Threading

Python's threading module runs multiple flows of execution within one process — essential for I/O-bound concurrency, and the foundation you need before tackling locks, thread pools, and the GIL's real implications.

A thread is an independent flow of execution sharing its parent process's memory. Python's threading module lets a single program do several things 'at once' — most usefully overlapping I/O waits — but understanding exactly what threads do and don't buy you in Python requires separating language-level threading behavior from CPython's specific GIL-driven execution model.

Process vs Thread

A process is an independent running program with its own memory space, file descriptors, and OS resources; processes do not share memory by default and communicating between them requires explicit mechanisms (pipes, sockets, shared memory). A thread is a lighter-weight unit of execution that lives inside a process and shares that process's memory with every other thread in it — all threads in one process see the same objects, the same global variables, the same heap.

This shared memory is threading's core trade-off: it makes communication between threads trivial (just read/write a shared variable) but makes safety hard (two threads mutating the same object concurrently can corrupt it, which is why locks and synchronization exist — covered in the next topic).

Process A                  Process B
+----------------+          +----------------+
| memory space A  |          | memory space B  |
| [thread 1]       |          | [thread 1]       |
| [thread 2]       |          +----------------+
| [thread 3]       |          (no shared memory
+----------------+           with process A)
threads inside one process
share the SAME memory

Why Threading in Python: I/O-Bound vs CPU-Bound

CPython implementation detail: CPython (up through its traditional, non-free-threaded builds) has a Global Interpreter Lock (GIL) — a single lock that ensures only one thread executes Python bytecode at any instant, even on a multi-core machine. This means Python threads in standard CPython do not achieve true CPU parallelism for pure-Python computation.

But the GIL is released during blocking I/O operations (network calls, file reads, time.sleep, and most C-extension calls that don't touch Python objects). This makes threading genuinely effective for I/O-bound work — many threads can be waiting on network responses or disk reads concurrently, with the GIL only briefly held by whichever thread is actively running Python bytecode. For CPU-bound work, multiprocessing (separate processes, real parallelism) is the usual answer instead — a separate topic.

PEP 703 introduced an experimental free-threaded build of CPython (available as an option since 3.13) that removes the GIL entirely, enabling true multi-core parallelism for threads. As of this writing it remains an evolving, opt-in build, not the default — but it's worth knowing this constraint is an implementation detail, not a permanent language guarantee.

Creating and Starting a Thread

The threading.Thread class wraps a target function to run on a separate thread. Creating a Thread object does not start it — you must call .start() explicitly.

start() returns immediately; the target function runs concurrently with the rest of the program from that point on.

import threading
import time

def worker(name: str, delay: float) -> None:
    print(f"{name} starting")
    time.sleep(delay)
    print(f"{name} finished")

t = threading.Thread(target=worker, args=("worker-1", 2))
t.start()          # begins running worker() concurrently
print("main thread continues immediately")

join: Waiting for a Thread to Finish

.join() blocks the calling thread until the target thread finishes. Without joining, the main program may exit (or move on) before spawned threads have completed their work.

Starting all threads first, then joining all of them, lets the three 1-second sleeps overlap — total wall time is about 1 second, not 3.

join() accepts an optional timeout; without one, it waits indefinitely.

import threading
import time

def worker(n: int) -> None:
    time.sleep(1)
    print(f"worker {n} done")

threads = [threading.Thread(target=worker, args=(i,)) for i in range(3)]
for t in threads:
    t.start()

for t in threads:
    t.join()          # wait for each thread to finish

print("all workers finished")

t = threading.Thread(target=worker, args=(0,))
t.start()
t.join(timeout=0.5)          # wait at most 0.5 seconds
if t.is_alive():
    print("still running after 0.5s")

Daemon Threads

By default, Python will not exit while any non-daemon thread is still running. A daemon thread is marked as disposable background work: the interpreter is allowed to exit even if daemon threads are still running, and they are killed abruptly (no cleanup) when that happens.

daemon must be set before start() is called (either via the Thread(daemon=True) argument or t.daemon = True). Use daemon threads for background housekeeping that doesn't need graceful shutdown; use non-daemon threads (the default) plus explicit stop signaling for work that must finish cleanly.

Because daemon threads are killed abruptly with no cleanup (no finally blocks guaranteed to run, no chance to flush buffered writes), never use one for anything that must complete safely, like writing a file or committing a transaction.

import threading
import time

def background_loop() -> None:
    while True:
        time.sleep(1)
        print("still working...")

t = threading.Thread(target=background_loop, daemon=True)
t.start()
time.sleep(3)
print("main thread exiting; daemon thread is killed automatically")
# program exits here even though background_loop() would otherwise run forever

Thread Identity

Every thread has a name and a numeric identifier, useful for logging and debugging concurrent programs.

import threading

def show_identity() -> None:
    current = threading.current_thread()
    print(f"name={current.name}, ident={current.ident}, native_id={threading.get_native_id()}")

t = threading.Thread(target=show_identity, name="worker-A")
t.start()
t.join()

print("main thread name:", threading.current_thread().name)   # 'MainThread'
print("active thread count:", threading.active_count())

Interruption and Cooperative Stopping

Python threads cannot be forcibly killed from outside (there is no safe thread.kill()) — abruptly terminating a thread mid-operation could leave shared data in a half-modified, inconsistent state. Instead, the standard pattern is cooperative: the thread's own loop periodically checks a shared flag (commonly a threading.Event) and exits voluntarily when asked.

threading.Event wraps a simple flag with thread-safe set()/clear()/is_set()/wait() methods — is_set() checks the flag, wait(timeout) blocks until it's set or the timeout elapses, whichever comes first.

import threading
import time

def worker(stop_event: threading.Event) -> None:
    while not stop_event.is_set():
        print("working...")
        time.sleep(0.5)
    print("stop requested, exiting cleanly")

stop_event = threading.Event()
t = threading.Thread(target=worker, args=(stop_event,))
t.start()

time.sleep(2)
stop_event.set()   # politely ask the thread to stop
t.join()            # wait for it to notice and exit

Passing Data and Sharing State

Threads in the same process share all global and heap-allocated objects, so passing 'results' out of a thread usually means writing to a shared container (a list, a queue, an object attribute) rather than a return value — Thread.start()/.join() has no built-in way to retrieve a return value directly.

This example previews synchronization (covered in depth in the Locks & Synchronization topic) — appending to a Python list is commonly safe without a lock due to CPython's GIL, but relying on that is fragile and non-portable; explicit locking is the correct, safe pattern.

import threading

results = []
lock = threading.Lock()

def compute(n: int) -> None:
    value = n * n
    with lock:                 # protect the shared list from concurrent append() races
        results.append(value)

threads = [threading.Thread(target=compute, args=(i,)) for i in range(5)]
for t in threads:
    t.start()
for t in threads:
    t.join()

print(sorted(results))   # [0, 1, 4, 9, 16]

Practical Usage: Concurrent I/O-Bound Downloads

Because each fetch spends its time blocked on (simulated) I/O — during which the GIL is released — the three 'downloads' overlap, giving roughly a 3x speedup over doing them sequentially.

import threading
import time

def fetch_url(url: str, results: dict, lock: threading.Lock) -> None:
    time.sleep(1)   # stand-in for a real network call
    with lock:
        results[url] = f"response from {url}"

urls = ["http://a.example", "http://b.example", "http://c.example"]
results: dict[str, str] = {}
lock = threading.Lock()

threads = [threading.Thread(target=fetch_url, args=(u, results, lock)) for u in urls]
start = time.monotonic()
for t in threads:
    t.start()
for t in threads:
    t.join()
print(f"fetched {len(results)} urls in {time.monotonic() - start:.1f}s")   # ~1.0s, not ~3.0s

Edge Cases

An unhandled exception inside a thread's target function does not propagate to the main thread or crash the program — it's printed to stderr and only that thread dies; the main thread and other threads continue running, which can silently hide bugs. Starting the same Thread object twice raises RuntimeError — a Thread instance can only be started once.

To surface exceptions from a worker thread, capture them explicitly (e.g. store the exception in a shared variable inside a try/except in the target function) rather than relying on Python to propagate them automatically.

import threading

def failing_worker() -> None:
    raise ValueError("something went wrong")

t = threading.Thread(target=failing_worker)
t.start()
t.join()
print("main thread is completely unaware the worker raised an exception")

t2 = threading.Thread(target=lambda: None)
t2.start()
t2.join()
try:
    t2.start()   # RuntimeError: threads can only be started once
except RuntimeError as e:
    print(e)

Mental Model and Quick Reference

Think of threads as workers sharing one desk (the process's memory): they can all reach the same papers (objects) at once, which is efficient for cooperation but requires discipline (locks) to avoid two workers writing over each other's work simultaneously. In CPython, think of the GIL as a single pen on that desk — only the worker holding the pen can actually write Python bytecode at a given instant, but a worker who's just waiting on the phone (I/O) puts the pen down for others to use.

Thread(target=f, args=(...)) -> create (not yet running)
.start()                       -> begin running concurrently
.join([timeout])                -> block until it finishes (or timeout)
.is_alive()                     -> still running?
.daemon = True                  -> killed automatically at interpreter exit
threading.Event()               -> cooperative stop-flag signaling
threading.current_thread()      -> info about the calling thread

Key Takeaways

Threads share their process's memory; processes do not — that's the core trade-off between them.

CPython's GIL (an implementation detail, not a language guarantee) means threads don't give CPU parallelism for pure Python code, but do give real concurrency for I/O-bound work because the GIL is released during blocking calls.

start() begins a thread; join() waits for it to finish; a Thread can only be started once.

Daemon threads are killed abruptly at interpreter exit with no cleanup — never use them for work that must finish safely.

There's no safe way to forcibly kill a thread; use a shared flag (commonly threading.Event) for cooperative stopping.

An exception inside a thread's target function does not propagate to the main thread — it must be captured and reported explicitly.

t = threading.Thread(target=worker) t.start() print("done") # main exits without waiting t = threading.Thread(target=worker) t.start() t.join() print("done")

Without join(), the main thread can proceed (or the program can exit) before the spawned thread has finished its work.

def worker(): while True: do_work() threading.Thread(target=worker, daemon=True).start() # later: how do we stop it cleanly? no mechanism provided def worker(stop_event): while not stop_event.is_set(): do_work() stop_event = threading.Event() threading.Thread(target=worker, args=(stop_event,)).start() # later: stop_event.set()

Relying on daemon=True to 'stop' a thread means it's killed abruptly with no cleanup at interpreter exit, not stopped safely; use a cooperative stop flag for anything that needs a clean shutdown.

try: threading.Thread(target=risky_task).start() except Exception as e: print("caught:", e) # never triggers def risky_task(errors: list): try: do_risky_thing() except Exception as e: errors.append(e) errors = [] t = threading.Thread(target=risky_task, args=(errors,)) t.start(); t.join() if errors: print("worker failed:", errors[0])

Exceptions raised inside a thread's target function do not propagate to the code that started the thread; they must be caught inside the target and reported through a shared variable.

t = threading.Thread(target=worker) t.start() t.start() # RuntimeError t = threading.Thread(target=worker) t.start() # create a NEW Thread object if you need to run it again t2 = threading.Thread(target=worker) t2.start()

A single Thread instance can only be started once; starting it again raises RuntimeError.

What is the key difference in memory sharing between processes and threads?

Threads within one process share that process's entire memory space (all global variables, heap objects); processes each have their own independent memory space and share nothing by default, requiring explicit IPC mechanisms to communicate.

Why does threading help with I/O-bound work in CPython despite the GIL?

The GIL is released during blocking I/O operations (network calls, file reads, sleeps), so while one thread waits on I/O, another thread is free to run Python bytecode — letting many I/O waits overlap even though only one thread executes Python code at any single instant.

What happens if you never call .join() on a thread you started?

The main program can proceed (or the interpreter can exit, for daemon threads) without waiting for that thread to finish, meaning its work may not be complete — or, for a non-daemon thread, the interpreter will still wait for it before fully exiting, but any code after start() in the main thread won't have waited for its results.

Why can't you forcibly kill a Python thread from outside it?

Abruptly terminating a thread mid-operation could leave shared data structures in a partially modified, inconsistent state (e.g. a lock left held, a list half-updated); Python instead relies on cooperative stopping, where the thread itself checks a shared flag and exits voluntarily.

What happens to a daemon thread when the main program is ready to exit?

It is killed abruptly with no cleanup — no finally blocks are guaranteed to run — as soon as no non-daemon threads remain, so daemon threads should never be used for work that must complete safely.

If a thread's target function raises an unhandled exception, does the main thread see it?

No — the exception is printed to stderr by default and only that thread terminates; the main thread and any other threads continue running unaffected and unaware, unless the exception is explicitly captured and communicated back (e.g. via a shared variable).

Is the GIL a permanent Python language guarantee?

No — it is a CPython implementation detail (present in CPython's traditional builds), not a feature of the Python language itself; PEP 703's free-threaded CPython build (available as an option since 3.13) removes it, though it is still an evolving, opt-in build rather than the default.

Write a function run_concurrently(tasks) that takes a list of zero-argument callables, runs each on its own thread, waits for all of them to finish, and returns nothing (fire-and-wait).

import threading def run_concurrently(tasks: list) -> None: threads = [threading.Thread(target=task) for task in tasks] for t in threads: t.start() for t in threads: t.join()

Threads must all be started first (so their work overlaps) and then joined in a separate loop; joining inside the same loop as starting would make each thread's join() block before the next one even starts, defeating the purpose of concurrency.

What is the practical problem with this code, beyond it printing normally?

A brand-new, never-set threading.Event() is passed in, so is_set() never becomes True and the worker spins forever consuming CPU in a tight busy-loop; since it's a daemon thread the interpreter can still exit, but the thread wastes CPU the whole time it's alive and there's no way given here to ever stop it cleanly.

Write a function that starts a worker thread, captures any exception it raises into a shared list, joins the thread, and re-raises the exception in the calling (main) thread if one occurred.

import threading def run_and_propagate(target, *args) -> None: errors = [] def wrapper(): try: target(*args) except Exception as e: errors.append(e) t = threading.Thread(target=wrapper) t.start() t.join() if errors: raise errors[0]

Since exceptions inside a thread's target don't propagate automatically, wrapper() catches any exception and stores it in a shared list; after joining, the main thread checks that list and re-raises, effectively surfacing the worker's failure to the caller.

import threading

def worker(stop_event):
    while not stop_event.is_set():
        pass  # busy loop, no sleep

t = threading.Thread(target=worker, args=(threading.Event(),), daemon=True)
t.start()
print("main thread continues")