Book Image

Python Parallel Programming Cookbook - Second Edition

By : Giancarlo Zaccone
Book Image

Python Parallel Programming Cookbook - Second Edition

By: Giancarlo Zaccone

Overview of this book

<p>Nowadays, it has become extremely important for programmers to understand the link between the software and the parallel nature of their hardware so that their programs run efficiently on computer architectures. Applications based on parallel programming are fast, robust, and easily scalable. </p><p> </p><p>This updated edition features cutting-edge techniques for building effective concurrent applications in Python 3.7. The book introduces parallel programming architectures and covers the fundamental recipes for thread-based and process-based parallelism. You'll learn about mutex, semaphores, locks, queues exploiting the threading, and multiprocessing modules, all of which are basic tools to build parallel applications. Recipes on MPI programming will help you to synchronize processes using the fundamental message passing techniques with mpi4py. Furthermore, you'll get to grips with asynchronous programming and how to use the power of the GPU with PyCUDA and PyOpenCL frameworks. Finally, you'll explore how to design distributed computing systems with Celery and architect Python apps on the cloud using PythonAnywhere, Docker, and serverless applications. </p><p> </p><p>By the end of this book, you will be confident in building concurrent and high-performing applications in Python.</p>

Related Content you might be interested in

Current Title:

Small book image
Python Parallel Programming Cookbook - Second Edition
Giancarlo Zaccone
Sep, 2019 | 370 pages
Small book image
Hands-On Data Structures and Algorithms with Python – Third Edition
Dr. Basant Agarwal
Jul, 2022 | 496 pages
Table of Contents (16 chapters)
Title Page
Title Page
Copyright and Credits
Copyright and Credits
Python Parallel Programming Cookbook Second Edition
Dedication
Dedication
About Packt
About Packt
Why subscribe?
Contributors
Contributors
About the author
About the reviewer
Packt is searching for authors like you
Preface
Preface
Who this book is for
What this book covers
To get the most out of this book
Sections
Get in touch
Free Chapter
1
Getting Started with Parallel Computing and Python
Getting Started with Parallel Computing and Python
Why do we need parallel computing?
Flynn's taxonomy
Memory organization
Parallel programming models
Evaluating the performance of a parallel program
Introducing Python
Introducing Python parallel programming
2
Thread-Based Parallelism
Thread-Based Parallelism
What is a thread?
Python threading module
Defining a thread
Determining the current thread
Defining a thread subclass
Thread synchronization with a lock
Thread synchronization with RLock
Thread synchronization with semaphores
Thread synchronization with a condition
Thread synchronization with an event
Thread synchronization with a barrier
Thread communication using a queue
3
Process-Based Parallelism
Process-Based Parallelism
Understanding Python's multiprocessing module 
Spawning a process 
Naming a process
Running processes in the background
Killing a process
Defining processes in a subclass
Using a queue to exchange data
Using pipes to exchange objects
Synchronizing processes
Using a process pool
4
Message Passing
Message Passing
Technical requirements
Understanding the MPI structure
Using the mpi4py Python module
Implementing point-to-point communication
Avoiding deadlock problems
Collective communication using a broadcast
Collective communication using the scatter function
Collective communication using the gather function
Collective communication using Alltoall
The reduction operation
Optimizing communication
5
Asynchronous Programming
Asynchronous Programming
Using the concurrent.futures Python module
Managing the event loop with asyncio
Handling coroutines with asyncio
Manipulating tasks with asyncio
Dealing with asyncio and futures
6
Distributed Python
Distributed Python
Introducing distributed computing
Types of distributed applications
Using the Python socket module
Distributed task management with Celery
RMI with Pyro4
7
Cloud Computing
Cloud Computing
What is cloud computing?
Understanding the cloud computing architecture
Developing web applications with PythonAnywhere
Dockerizing a Python application
Introducing serverless computing
8
Heterogeneous Computing
Heterogeneous Computing
Understanding heterogeneous computing
Understanding the GPU architecture
Understanding GPU programming
Dealing with PyCUDA
Heterogeneous programming with PyCUDA
Implementing memory management with PyCUDA
Introducing PyOpenCL
Building applications with PyOpenCL
Element-wise expressions with PyOpenCL
Evaluating PyOpenCL applications
GPU programming with Numba
9
Python Debugging and Testing
Python Debugging and Testing
What is debugging?
What is software testing?
Debugging using Winpdb Reborn
Interacting with pdb
Implementing rpdb for debugging
Dealing with unittest
Application testing using nose
Other Books You May Enjoy
Other Books You May Enjoy
Leave a review - let other readers know what you think

Collective communication using Alltoall

The Alltoall collective communication combines the scatter and gather functionalities.

How to do it...

In the following example, we'll see an mpi4py implementation of comm.Alltoall. We'll consider a communicator a group of processes, where each process sends and receives an array of numerical data from the other processes defined in the group:

  1. For this example, the relevant mpi4py and numpy libraries must be imported:
from mpi4py import MPI 
import numpy
  1. As in the previous example, we need to set the same parameters, comm, size, and rank:
comm = MPI.COMM_WORLD 
size = comm.Get_size() 
rank = comm.Get_rank()
  1. Hence, we must define the data that each process...
Previous Section
End of Section 10
Next Section