Concurrency in Python: Understanding Multithreading, Multiprocessing, and Coroutines

In a world where speed and responsiveness matter, concurrency becomes a crucial concept for developers. Whether you’re building a web server, processing data, or scraping websites, making your program do multiple things at once can greatly improve performance.

Python offers three primary approaches to concurrency:

• Multithreading
• Multiprocessing
• Coroutines (asyncio)

Each has unique strengths and use cases. In this article, we’ll break them down with code examples, use cases, and comparisons so you can choose the right tool for the job.

---

What is Concurrency?

Concurrency is the ability of a program to manage multiple tasks at once. It’s not the same as parallelism (which involves executing tasks simultaneously on multiple cores), but it’s related.

In Python, due to the Global Interpreter Lock (GIL), understanding how concurrency works is key to optimizing performance.

🧵 1. Python Multithreading

What is Multithreading?

Multithreading allows multiple threads (smaller units of a process) to run concurrently within a single Python process. Useful for I/O-bound tasks like:

• File reading/writing
• Network calls
• Web scraping

import threading
import time

def download_file(file_id):
    print(f"Start downloading {file_id}")
    time.sleep(2)
    print(f"Finished downloading {file_id}")

threads = []

for i in range(5):
    t = threading.Thread(target=download_file, args=(i,))
    threads.append(t)
    t.start()

for t in threads:
    t.join()

Pros:

• Low memory usage (single process)
• Great for I/O-bound tasks

Cons:

• Not truly parallel due to GIL
• Bad choice for CPU-bound tasks

🧠 2. Python Multiprocessing

What is Multiprocessing?

Multiprocessing creates separate Python processes, each with its own GIL and memory space. Ideal for CPU-bound tasks like:

• Data processing
• Image manipulation
• Machine learning

from multiprocessing import Process
import os

def task(name):
    print(f"Running task {name} in PID: {os.getpid()}")

if __name__ == "__main__":
    processes = []
    for i in range(4):
        p = Process(target=task, args=(f"Task-{i}",))
        processes.append(p)
        p.start()

    for p in processes:
        p.join()

Pros:

• True parallelism
• Utilizes multiple CPU cores
• Ideal for heavy computations

Cons:

• Higher memory usage
• Slower startup (process creation overhead)
• Data sharing is complex

⚡ 3. Coroutines with asyncio

What is asyncio?

asyncio is Python’s built-in library for writing asynchronous, non-blocking code using coroutines. It’s best for high-performance I/O applications like:

• Web servers (FastAPI, AIOHTTP)
• Real-time APIs
• Concurrent HTTP requests

import asyncio

async def download_file(file_id):
    print(f"Start downloading {file_id}")
    await asyncio.sleep(2)
    print(f"Finished downloading {file_id}")

async def main():
    tasks = [download_file(i) for i in range(5)]
    await asyncio.gather(*tasks)

asyncio.run(main())

Pros:

• Lightweight, scalable
• High performance for I/O tasks
• Clean syntax with async/await

Cons:

• Learning curve
• Not suitable for CPU-heavy work
• Blocking code can freeze the entire loop

🔍 Comparison Table

Feature	Multithreading	Multiprocessing	Coroutines (asyncio)
GIL Bypassed?	❌ No	✅ Yes	❌ No
I/O Bound	✅ Excellent	✅ Good	✅ Excellent
CPU Bound	❌ Poor	✅ Excellent	❌ Poor
Memory Usage	✅ Low	❌ High	✅ Low
Parallelism	❌ Simulated	✅ True	❌ Simulated
Complexity	✅ Easy	⚠️ Medium	⚠️ Medium

🧠 When to Use What?

• Use multithreading for I/O-heavy tasks when you want simple concurrency (e.g., downloading files).
• Use multiprocessing for CPU-intensive tasks that require real parallel processing (e.g., image processing, ML training).
• Use asyncio when building highly-scalable, asynchronous I/O systems (e.g., chat servers, APIs).

🛠 Hybrid Approaches

Python allows mixing these approaches. For example:

• Use asyncio with thread pools for blocking I/O.
• Use multiprocessing with queues to offload CPU-bound tasks from an asyncio loop.

import asyncio
from concurrent.futures import ThreadPoolExecutor

def blocking_task(x):
    return x * 2

async def main():
    loop = asyncio.get_event_loop()
    with ThreadPoolExecutor() as pool:
        result = await loop.run_in_executor(pool, blocking_task, 10)
        print(result)

asyncio.run(main())

Conclusion

Concurrency in Python is not one-size-fits-all. Depending on the nature of your task — I/O or CPU-bound — you can choose from multithreading, multiprocessing, or asyncio.

Mastering these tools will help you write efficient, scalable, and performant Python applications.