Thread Pools

concurrent.futures.ThreadPoolExecutor manages a fixed pool of worker threads for you — submit work, get back Future objects, and stop hand-rolling thread creation and joining for every concurrent task.

Creating a new thread for every unit of work doesn't scale — thread creation has overhead, and an unbounded number of threads can overwhelm a system. concurrent.futures.ThreadPoolExecutor solves this by maintaining a fixed, reusable pool of worker threads and a clean, Future-based API for submitting work and collecting results.

Why a Thread Pool Instead of Manual Threads

The Threading topic showed creating one threading.Thread per task, then manually tracking and joining each one. That approach breaks down at scale: creating a thread has real overhead, an unbounded number of concurrent threads can exhaust system resources, and manually collecting results and exceptions from many threads is tedious and error-prone.

concurrent.futures.ThreadPoolExecutor (standard library, concurrent.futures module) fixes all of this: a bounded number of worker threads are created once and reused across many submitted tasks, and each submitted task returns a Future object representing its eventual result — a much cleaner abstraction than manually managing Thread objects.

Creating a ThreadPoolExecutor and submit()

submit(fn, *args, **kwargs) schedules a callable to run on the pool and immediately returns a Future — a placeholder for a result that isn't ready yet.

The executor is a context manager: exiting the with block calls shutdown(wait=True), blocking until all submitted work finishes before the block exits.

from concurrent.futures import ThreadPoolExecutor
import time

def slow_square(n: int) -> int:
    time.sleep(1)
    return n * n

with ThreadPoolExecutor(max_workers=4) as executor:
    future = executor.submit(slow_square, 5)
    print(future.done())          # probably False — it's still running
    result = future.result()      # blocks until the result is ready
    print(result)                  # 25

The Future Object

A Future represents work that may not have finished yet. Its key methods: .result(timeout=None) blocks until the callable returns (or raises whatever exception the callable raised, or raises TimeoutError if the timeout elapses first); .done() checks completion without blocking; .exception() retrieves a raised exception without re-raising it; .add_done_callback(fn) registers a function to run automatically when the future completes.

Unlike a raw threading.Thread, exceptions raised inside the submitted function ARE captured and re-raised when you call .result() — this is one of ThreadPoolExecutor's biggest ergonomic advantages.

from concurrent.futures import ThreadPoolExecutor

def might_fail(n: int) -> int:
    if n == 0:
        raise ValueError("cannot process zero")
    return 100 // n

with ThreadPoolExecutor(max_workers=2) as executor:
    future = executor.submit(might_fail, 0)
    try:
        result = future.result()
    except ValueError as e:
        print("task raised:", e)     # exceptions propagate through .result()

submit() for Many Independent Tasks

With max_workers=3 and 5 tasks, only 3 run concurrently at once; the pool automatically starts the next queued task as soon as a worker thread frees up.

from concurrent.futures import ThreadPoolExecutor

def fetch(url: str) -> str:
    return f"content of {url}"   # stand-in for a real network call

urls = [f"http://example.com/{i}" for i in range(5)]

with ThreadPoolExecutor(max_workers=3) as executor:
    futures = [executor.submit(fetch, url) for url in urls]
    for future in futures:
        print(future.result())

map(): The Simpler Alternative to submit()

executor.map(fn, iterable) is a higher-level convenience: it applies fn to every item, dispatching across the pool, and returns an iterator of results in the same order the inputs were given — closely mirroring the built-in map(), but concurrent.

map() results preserve input order regardless of which task finishes first internally — the 3rd result always corresponds to the 3rd input, even if it happened to finish before the 1st.

Use map() when you're applying the same function to a batch of inputs and want results in order; use submit() when tasks are heterogeneous, you need individual Future objects, or you want to process results as they complete rather than in submission order.

from concurrent.futures import ThreadPoolExecutor
import time

def slow_square(n: int) -> int:
    time.sleep(1)
    return n * n

with ThreadPoolExecutor(max_workers=4) as executor:
    results = list(executor.map(slow_square, [1, 2, 3, 4]))
    print(results)   # [1, 4, 9, 16] — takes about 1 second total, not 4

Processing Results as They Complete: as_completed

concurrent.futures.as_completed(futures) yields each Future as soon as it finishes, regardless of submission order — useful when you want to react to the fastest results first rather than waiting for a specific one.

Mapping each Future back to its original input (via a dict keyed by the Future object) is the standard way to know which submitted task a given completed Future corresponds to.

from concurrent.futures import ThreadPoolExecutor, as_completed
import time
import random

def task(n: int) -> int:
    time.sleep(random.uniform(0.1, 1.0))
    return n

with ThreadPoolExecutor(max_workers=5) as executor:
    futures = {executor.submit(task, i): i for i in range(5)}
    for future in as_completed(futures):
        original_input = futures[future]
        print(f"task {original_input} finished with result {future.result()}")

Cancellation

future.cancel() attempts to cancel a pending task, but only succeeds if the task hasn't started running yet — once a worker thread has begun executing the callable, cancel() returns False and has no effect, because Python cannot forcibly interrupt a running thread (as covered in the Threading topic).

Because f2 never started running, cancelling it works and its callable never executes at all; f1 was already underway and must be allowed to finish (or its own code must cooperatively check for a stop signal, per the Threading topic's pattern).

from concurrent.futures import ThreadPoolExecutor
import time

def slow_task() -> str:
    time.sleep(2)
    return "done"

with ThreadPoolExecutor(max_workers=1) as executor:
    f1 = executor.submit(slow_task)     # starts running immediately (only worker)
    f2 = executor.submit(slow_task)     # queued, waiting for the one worker

    time.sleep(0.1)
    print(f1.cancel())   # False — already running, can't be cancelled
    print(f2.cancel())   # True — still queued, successfully cancelled

shutdown() and Executor Lifecycle

shutdown(wait=True) (called automatically when exiting a with block) tells the executor to stop accepting new tasks and, with wait=True (the default), blocks until all already-submitted tasks finish. shutdown(cancel_futures=True) (Python 3.9+) additionally cancels any tasks that haven't started yet.

Prefer the with ThreadPoolExecutor(...) as executor: form over manual shutdown() calls — it guarantees cleanup even if an exception is raised while submitting or processing work.

from concurrent.futures import ThreadPoolExecutor

executor = ThreadPoolExecutor(max_workers=2)
futures = [executor.submit(lambda: 1 + 1) for _ in range(5)]
executor.shutdown(wait=True)     # blocks until all 5 tasks complete
print([f.result() for f in futures])   # all results available now

Choosing max_workers

For I/O-bound work (the typical use case for a thread pool, given the GIL — see the Threading topic), max_workers can reasonably exceed the number of CPU cores, since threads spend most of their time waiting on I/O rather than competing for CPU. Since Python 3.8, the default max_workers (when omitted) is min(32, os.cpu_count() + 4), a reasonable general-purpose default for I/O-bound workloads.

For CPU-bound work, a thread pool will NOT give parallel speedup in standard CPython due to the GIL — use concurrent.futures.ProcessPoolExecutor (a separate, related executor using real OS processes) instead, which is outside this topic's scope.

from concurrent.futures import ThreadPoolExecutor

# Fine to set higher than CPU count for I/O-bound work (network calls, disk reads)
with ThreadPoolExecutor(max_workers=20) as executor:
    pass

# Omit max_workers to use the sensible library default
with ThreadPoolExecutor() as executor:
    pass

Practical Usage: Concurrent Downloads with Error Handling

This pattern — dict mapping Future to identifying context, iterate with as_completed, catch exceptions per-future — is the standard way to run many independent, fallible tasks concurrently and collect both successes and failures.

from concurrent.futures import ThreadPoolExecutor, as_completed

def download(url: str) -> str:
    if "bad" in url:
        raise ConnectionError(f"failed to reach {url}")
    return f"downloaded {url}"

urls = ["http://a.example", "http://bad.example", "http://c.example"]

results = {}
errors = {}

with ThreadPoolExecutor(max_workers=3) as executor:
    future_to_url = {executor.submit(download, url): url for url in urls}
    for future in as_completed(future_to_url):
        url = future_to_url[future]
        try:
            results[url] = future.result()
        except ConnectionError as e:
            errors[url] = str(e)

print(results)
print(errors)

Edge Cases

Calling .result() on a Future whose task raised an exception re-raises that exception at the call site — always wrap it in try/except when the task can fail. map() propagates the first exception encountered when you iterate its results (lazily, as you consume the iterator) rather than immediately when map() is called. Submitting work after the executor has been shut down raises RuntimeError.

from concurrent.futures import ThreadPoolExecutor

def divide(a: int, b: int) -> float:
    return a / b

with ThreadPoolExecutor() as executor:
    pass

try:
    executor.submit(divide, 1, 1)   # RuntimeError: executor already shut down
except RuntimeError as e:
    print(e)

Key Takeaways

ThreadPoolExecutor manages a bounded, reusable pool of worker threads instead of creating a new thread per task.

submit() returns a Future immediately; .result() blocks until the task finishes and re-raises any exception the task raised.

map() applies a function across an iterable concurrently and returns results in input order; as_completed() yields Futures in the order they actually finish.

cancel() only succeeds for tasks that haven't started running yet — a running task cannot be forcibly stopped.

Always use the executor as a context manager (with ThreadPoolExecutor(...) as executor:) so shutdown happens automatically and reliably.

Thread pools help I/O-bound work in CPython due to the GIL being released during I/O; for CPU-bound work, use ProcessPoolExecutor instead.

future = executor.submit(risky_task) #.result() never called — exception silently swallowed future = executor.submit(risky_task) try: result = future.result() except Exception as e: print("task failed:", e)

An exception raised inside a submitted task is stored on the Future and only surfaces when .result() (or .exception()) is called; if nothing ever calls it, the failure goes unnoticed.

executor = ThreadPoolExecutor(max_workers=4) for url in urls: executor.submit(fetch, url) # forgot shutdown/with — program may exit before tasks finish, or resources leak with ThreadPoolExecutor(max_workers=4) as executor: for url in urls: executor.submit(fetch, url) # shutdown(wait=True) runs automatically here

Using the executor as a context manager guarantees shutdown(wait=True) runs, ensuring all submitted work completes and resources are released even if an exception occurs.

future.cancel() # assumed to stop a task that's already running if not future.cancel(): print("already running; use a cooperative stop flag if it needs to be interruptible") cancel() only works on tasks that haven't started yet; a running task must check its own cooperative stop signal (as in the Threading topic) — cancel() cannot forcibly interrupt it.

with ThreadPoolExecutor(max_workers=200) as executor: # CPU-bound work, expecting speedup from more threads executor.map(cpu_heavy_function, data) from concurrent.futures import ProcessPoolExecutor with ProcessPoolExecutor() as executor: executor.map(cpu_heavy_function, data)

A thread pool cannot parallelize CPU-bound pure-Python work in standard CPython due to the GIL; use ProcessPoolExecutor (real OS processes) for CPU-bound parallelism instead.

What advantage does submit() have over manually creating and starting a threading.Thread for a task that might raise an exception?

The Future returned by submit() captures any exception the task raises and re-raises it when .result() is called, giving clean, centralized error handling — whereas an exception in a raw Thread's target function is silently printed to stderr and never propagates back to the code that started the thread.

What is the difference between executor.map() and using as_completed() with submit()?

map() returns results in the same order as the input iterable, regardless of which task actually finishes first; as_completed() yields each Future as soon as it finishes, in completion order, which can differ from submission order — useful when you want to react to the fastest results first.

Under what condition does future.cancel() actually succeed?

Only if the task has not yet started executing on a worker thread; once a worker has begun running the callable, cancel() returns False and has no effect, because Python provides no safe way to forcibly interrupt a running thread.

Why is a thread pool a good fit for I/O-bound work but not CPU-bound work in standard CPython?

The GIL prevents true parallel execution of Python bytecode across threads, but it is released during blocking I/O, so I/O-bound tasks overlap productively; CPU-bound pure-Python tasks would still be serialized by the GIL, giving no speedup (ProcessPoolExecutor, using separate processes, is the right tool for CPU-bound parallelism instead).

What happens when you call executor.submit() after the executor has already been shut down?

It raises RuntimeError — a shut-down executor no longer accepts new work.

Why is with ThreadPoolExecutor(...) as executor: preferred over manually calling shutdown()?

The context manager guarantees shutdown(wait=True) runs even if an exception occurs while submitting or processing tasks, ensuring all in-flight work completes and the pool's resources are reliably released.

Write a function fetch_all(urls) that uses a ThreadPoolExecutor with max_workers=5 to call a given fetch(url) function concurrently for every URL, returning a dict mapping each url to its result, and mapping to None for any url whose fetch raised an exception.

from concurrent.futures import ThreadPoolExecutor, as_completed def fetch_all(urls: list[str], fetch) -> dict: results = {} with ThreadPoolExecutor(max_workers=5) as executor: future_to_url = {executor.submit(fetch, url): url for url in urls} for future in as_completed(future_to_url): url = future_to_url[future] try: results[url] = future.result() except Exception: results[url] = None return results

Each submitted Future is mapped back to its originating URL via a dict; as_completed() processes them as they finish, and a per-future try/except ensures one failing URL doesn't stop the others from being collected.

What does this print, and roughly how long does it take to run?

It prints [2, 4, 6, 8], preserving input order. With max_workers=2 and 4 one-second tasks, they run in two batches of two concurrent tasks each, so the total time is roughly 2 seconds, not 4.

Write code that submits 3 tasks to a ThreadPoolExecutor with only 1 worker, then demonstrates that a still-queued task can be cancelled but a currently-running one cannot.

from concurrent.futures import ThreadPoolExecutor import time def slow(): time.sleep(1) return "done" with ThreadPoolExecutor(max_workers=1) as executor: f1 = executor.submit(slow) # starts immediately (the one worker) f2 = executor.submit(slow) # queued f3 = executor.submit(slow) # queued time.sleep(0.1) # give f1 a moment to actually start running print("f1 cancel:", f1.cancel()) # False, already running print("f2 cancel:", f2.cancel()) # True, still queued print("f3 cancel:", f3.cancel()) # True, still queued

With max_workers=1, f1 begins executing immediately while f2 and f3 wait in the pool's internal queue; a short sleep lets f1 actually start before checking, after which f1.cancel() fails (already running) while f2.cancel() and f3.cancel() succeed because they never started.

from concurrent.futures import ThreadPoolExecutor
import time

def task(n):
    time.sleep(1)
    return n * 2

with ThreadPoolExecutor(max_workers=2) as executor:
    results = list(executor.map(task, [1, 2, 3, 4]))
print(results)