Asynchronous programming with AsyncIO

Before we dive into this unknown territory of asynchronous programming, let's first try to recall why we used threads or multiple processes.

One of the main reasons to use threads or multiple processes was to increase the concurrency and, as a result, the ability of the application to handle a higher number of concurrent requests. But this came at a cost of increased resource utilization, and the limited ability to run multiple threads or the launching of heavier processes to accommodate higher concurrency with complex mechanisms of implementing locks between the shared data structures.

Now, in the context of building a scalable web application, we also have a few major differences from a general purpose compute-heavy application. In the functioning of the web application, most of the time, the application is waiting on the I/O so as to receive the complete request or to complete the sending of the response to the client. This was the part where we exploit the use of multiple threads since, at any particular point time, some of the threads can be in a waiting state to complete the I/O and that time can be used in processing other requests by the application.

But we can have something better here. How about if we could use a single thread that executes a task and then, as soon as the task goes into a waiting state (doing some disk I/O or network I/O), the thread swaps out the task for another task. This continues until all the tasks inside the queue have finished executing. This process of asynchronously executing the tasks without really waiting for the results of one task to arrive before calling in the next task is known as asynchronous programming.

A simple example of asynchronous programming might involve a web application which makes a call to three different NEWS feeds, namely site1, site2, and site3. With asynchronous programming, once our program starts executing, it will make a call to site1 and without waiting for the results to arrive, it will start the execution of a call to site2, and so on to site3. Since the order in which the results will start arriving can be different anytime, the final output also may differ in the multiple runs of the same program.

This allows us better utilization of the resources while also running the program in a single-threaded mode, where the multiple tasks are multiplexed over a single thread only and are checked periodically for their completion.

Python introduced the support for asynchronous programming by making available the AsyncIO package inside the Python distributions. The way AsyncIO works is through the implementation of an event loop and co-routines.

But, before we dive into the implementation of programs with the Python AsyncIO, let's spend some time understanding the terminology associated with the framework.