-
Book Overview & Buying
-
Table Of Contents
-
Feedback & Rating

Learning Concurrency in Python
By :

Now that we've had an in-depth look at both ThreadPoolExecutors
and ProcessPoolExecutors
, it's time to actually put these newly learned concepts into practice. In Chapter 5, Communication between Threads, we started developing a multithreaded web crawler that was able to crawl every available link on a given website.
The full source code for this Python web crawler can be found at this link: https://github.com/elliotforbes/python-crawler.
It didn't, however, output the results in the most readable format, and the code could be improved using ThreadPoolExecutors
. So, let's have a look at implementing both more readable code and more readable results.
Before we get started, we need to define a general plan as to how we are going to improve our crawler.
A few examples of the improvements we might wish to make are as follows:
ThreadPoolExecutors