Python Multithreading Examples: Thread, Lock, Queue, and ThreadPoolExecutor

Learn Python multithreading with practical examples using threading.Thread, start(), join(), daemon threads, Lock, Queue, ThreadPoolExecutor, and best practices for I/O-bound tasks.

Published

Updated

Read time 7 min read

Reviewed byDeepak Prasad

Python Multithreading Examples: Thread, Lock, Queue, and ThreadPoolExecutor

The threading module runs multiple threads inside one process: they share memory, so passing data between them is straightforward. The standard library also documents ThreadPoolExecutor in concurrent.futures as a higher-level way to run callables asynchronously with a pool of threads. This guide follows that practical path—Thread, start, join, daemon threads, Lock, Queue, and ThreadPoolExecutor—not a full catalog of every synchronization primitive.

For CPU-bound parallelism across cores in CPython, see Python multiprocessing. For structured error handling in ordinary code paths, see Python try except.

Tested on: Python 3.13.3; kernel 6.14.0-37-generic.

Thread output order and timing can differ between machines; snippets use {run=false} so you can run them as python script.py locally without relying on an in-browser runner.


What is multithreading in Python?

A thread is a separate flow of execution inside the same OS process. Python’s threading module lets you create Thread objects, each running a callable (usually a function) concurrently with the rest of your program. Because threads share one address space, they can read and write the same Python objects—unlike processes, which need pickling or IPC.

In CPython, the Global Interpreter Lock (GIL) allows only one thread to execute Python bytecode at a time, so multiple threads do not speed up tight CPU-bound pure-Python loops the way multiple processes can. Threads still help when work spends time waiting on I/O, because many blocking calls release the GIL while they wait, letting other threads run.


When should you use Python multithreading?

Use threads when:

  • Tasks are I/O-bound (network clients, blocking file reads, database drivers that release the GIL while waiting).
  • You want overlapped waiting without rewriting everything as async await.

Avoid expecting threads to parallelize heavy numeric Python on all cores; for that, use multiprocessing or native extensions / vectorized libraries that release the GIL during compute.


Python multithreading vs multiprocessing

Topic Multithreading Multiprocessing
Memory Shared in one process Separate per process
GIL One bytecode thread at a time for typical Python Each process has its own interpreter and GIL
Best for Overlapping blocking I/O CPU-bound pure Python across cores
Data sharing Shared objects (with locks where needed) Pickle / queues / explicit shared memory

Basic Python multithreading example

Create a Thread with target= set to the function to run, then call start():

python
import threading


def greet():
    print("hello from thread")


if __name__ == "__main__":
    t = threading.Thread(target=greet)
    t.start()
    print("hello from main")

Both messages can appear; their order is not guaranteed without synchronization.


Start and join threads

start() begins execution in the background. join() blocks until the thread completes (or until timeout= seconds pass).

python
import threading
import time


def work():
    time.sleep(0.05)
    print("worker done")


if __name__ == "__main__":
    t = threading.Thread(target=work)
    t.start()
    t.join()
    print("main after join")

Here main after join always prints after worker done because the main thread waits for t.


Pass arguments to a thread

Pass positional arguments with args= (a tuple) and keywords with kwargs=:

python
import threading


def add(a, b, label="sum"):
    print(label, a + b)


if __name__ == "__main__":
    t = threading.Thread(target=add, args=(2, 3), kwargs={"label": "result"})
    t.start()
    t.join()

Run multiple threads in a loop

Start several threads, then join them so the program does not exit early:

python
import threading


def show(n):
    print("thread", n)


if __name__ == "__main__":
    threads = [threading.Thread(target=show, args=(i,)) for i in range(3)]
    for t in threads:
        t.start()
    for t in threads:
        t.join()
    print("all joined")

Print order may still be interleaved; join only guarantees completion, not ordering.


Daemon thread in Python

Pass daemon=True so the interpreter will not block process exit on that thread. Daemons are appropriate for background housekeeping, not for work that must finish before shutdown.

python
import threading
import time


def background():
    time.sleep(0.2)
    print("daemon finishing")


if __name__ == "__main__":
    t = threading.Thread(target=background, daemon=True)
    t.start()
    print("main exits immediately unless we join")

If the main thread reaches the end of the script right after start() without join(), the process can exit before the daemon prints anything—always join() when you need its output or side effects.


Use Lock to avoid race conditions

When two threads mutate the same mutable state, protect critical sections with threading.Lock (acquire / release, or the with lock: pattern):

python
import threading

counter = 0
lock = threading.Lock()


def bump():
    global counter
    for _ in range(100_000):
        with lock:
            counter += 1


if __name__ == "__main__":
    t1 = threading.Thread(target=bump)
    t2 = threading.Thread(target=bump)
    t1.start()
    t2.start()
    t1.join()
    t2.join()
    print(counter)

You should see 200000. Without the lock, races can lose increments.


Share data between threads using Queue

queue.Queue is designed for producer–consumer patterns between threads: put and get are synchronized.

python
import threading
import queue


def producer(q, values):
    for v in values:
        q.put(v)


def consumer(q, n, out):
    for _ in range(n):
        out.append(q.get())
    q.task_done()


if __name__ == "__main__":
    q = queue.Queue()
    results = []
    t1 = threading.Thread(target=producer, args=(q, [1, 2, 3]))
    t2 = threading.Thread(target=consumer, args=(q, 3, results))
    t1.start()
    t2.start()
    t1.join()
    t2.join()
    print(sorted(results))

Use ThreadPoolExecutor

concurrent.futures.ThreadPoolExecutor manages a pool of worker threads and a work queue. Prefer it over manual thread lists when you have many similar jobs.

python
from concurrent.futures import ThreadPoolExecutor


def square(n):
    return n * n


if __name__ == "__main__":
    with ThreadPoolExecutor(max_workers=3) as ex:
        futures = [ex.submit(square, i) for i in range(4)]
        print([f.result() for f in futures])

Using the executor as a context manager calls shutdown(wait=True) when the block ends.

ThreadPoolExecutor submit() example

submit(fn, *args) returns a Future; call .result() to block for the return value (or raise the exception raised inside the worker).

python
from concurrent.futures import ThreadPoolExecutor


def slow_add(a, b):
    return a + b


if __name__ == "__main__":
    with ThreadPoolExecutor(max_workers=2) as ex:
        fut = ex.submit(slow_add, 40, 2)
        print(fut.result())

ThreadPoolExecutor map() example

map(func, iterable) applies func to each item, preserving input order in the results iterator even if tasks finish out of order.

python
from concurrent.futures import ThreadPoolExecutor


def label(n):
    return f"item-{n}"


if __name__ == "__main__":
    with ThreadPoolExecutor(max_workers=3) as ex:
        print(list(ex.map(label, [2, 0, 1])))

Handle exceptions in threads

Exceptions raised inside a Thread target do not propagate to the caller of start(); the thread object records the exception (Python 3.8+: threading.excepthook runs for uncaught thread exceptions). Handle errors inside the target, or use Future.result() from ThreadPoolExecutor, which re-raises the worker exception in the calling thread.

python
import threading


def risky():
    raise RuntimeError("boom")


if __name__ == "__main__":
    t = threading.Thread(target=risky)
    t.start()
    t.join()
    print("thread ended; check logs / excepthook for traceback")

With ThreadPoolExecutor:

python
from concurrent.futures import ThreadPoolExecutor, as_completed


def maybe_bad(n):
    if n == 1:
        raise ValueError("bad")
    return n * 2


if __name__ == "__main__":
    with ThreadPoolExecutor(max_workers=2) as ex:
        futs = [ex.submit(maybe_bad, i) for i in (0, 1, 2)]
        for f in as_completed(futs):
            try:
                print(f.result())
            except ValueError as e:
                print("caught", e)

Common Python multithreading mistakes

  • Expecting true parallel CPU for pure Python loops on all cores—threads share the GIL; use multiprocessing or native code for that shape of work.
  • Mutating shared globals without a lock or other synchronization—race conditions corrupt counters, lists, and dicts.
  • Using daemon threads for work that must complete before process exit, then exiting the main thread immediately.
  • Calling run() instead of start() by mistake—run() executes synchronously in the current thread; start() launches a new thread.
  • Ignoring exceptions in threads—errors disappear into stderr unless you handle them in the target or use futures / hooks. For structured handling, see Python try except.

Python multithreading best practices

  • Prefer ThreadPoolExecutor when the work maps cleanly to “run this callable many times.”
  • Keep shared mutable state small and guarded with locks, or pass messages through Queue.
  • Use join() (or Future.result()) when you need completion or ordered shutdown.
  • For CPU profiling of hot paths, measure with Python measure execution time and consider the right concurrency model before tuning thread counts.

Python multithreading quick reference table

Goal API
Background function threading.Thread(target=fn, args=..., kwargs=...)
Wait for completion thread.join()
Non-blocking exit Thread(..., daemon=True) (use carefully)
Protect shared mutable state threading.Lock() and with lock:
Producer / consumer queue.Queue
Pool of workers ThreadPoolExecutor(max_workers=n)
One-shot async result executor.submit(fn, *args).result()
Ordered parallel map list(executor.map(fn, iterable))

Summary

Python multithreading is about overlapping I/O and blocking calls inside one process: Thread plus start and join is the low-level pattern, while Lock and Queue keep shared data safe. Daemon threads are for best-effort background work only. For many similar jobs, ThreadPoolExecutor with submit or map is usually clearer than manual thread lists. Remember that the GIL limits parallel CPU-bound Python bytecode; reach for multiprocessing or other tools when that is your bottleneck, and handle or surface thread exceptions explicitly so failures are visible.


References


Frequently Asked Questions

1. What is multithreading in Python?

Multithreading runs several threads of control inside one process; they share memory, so communication is cheap, but CPython’s GIL means CPU-bound Python bytecode does not execute in parallel across threads the way separate processes can.

2. When should I use threads instead of asyncio or multiprocessing?

Use threads for blocking I/O you want to overlap (network, disk APIs that release the GIL); use asyncio when libraries expose async/await; use multiprocessing for CPU-heavy pure Python work that needs multiple cores.

3. What is the difference between start() and join() on a Thread?

start() asks the interpreter to begin running the thread’s target in the background; join() blocks the caller until that thread finishes (or until an optional timeout elapses).

4. What is a daemon thread?

A daemon thread is marked with daemon=True so the process can exit without waiting for it; use it for background helpers, never for work that must finish before shutdown.

5. When should I use ThreadPoolExecutor?

Use concurrent.futures.ThreadPoolExecutor when you have many short callable jobs you want to run across a bounded pool of worker threads with a simple submit or map API instead of managing Thread objects by hand.
Deepak Prasad

R&D Engineer

Founder of GoLinuxCloud with more than 15 years of expertise in Linux, Python, Go, Laravel, DevOps, Kubernetes, Git, Shell scripting, OpenShift, AWS, Networking, and Security. With extensive …