Orientation and Essential Vocabulary

Learning Objectives

Define concurrency, parallelism, asynchronous programming, multithreading, multiprocessing, non-blocking I/O and event loops precisely — and tell them apart.

Distinguish synchronous/asynchronous, blocking/non-blocking, sequential/concurrent, CPU-bound/I/O-bound, latency/throughput and responsiveness/total time.

Name the units of execution — processes, threads, tasks, coroutines — and the two scheduling styles, cooperative and preemptive.

Compare shared-memory and message-passing concurrency, and explain why nondeterminism makes concurrency hard.

Read the course's code and output conventions.

Plain-Language Explanation

A restaurant kitchen is a good model. Concurrency is one cook juggling three orders: while the pasta boils, they chop salad, then check the oven — all three orders *make progress* during the same half hour, even though the cook does one thing at a time. Parallelism is three cooks each cooking an order *at the same instant*. Asynchronous work is putting the pasta on to boil and setting a timer instead of standing and watching the pot: you start something, go do other work, and get notified when it is done.

Computers do the same. A web server handles thousands of clients concurrently; a video encoder uses eight CPU cores in parallel; a phone app fetches data asynchronously so the screen keeps responding. The difficulty is that when several things happen in overlapping time, they can interfere — two cooks grabbing the same pan — and the exact order of events changes from run to run.

Precise Technical Explanation

What is concurrency?

Concurrency is the composition of independently progressing activities whose lifetimes overlap. It is a property of program *structure*: a concurrent program is designed so that several tasks can be in progress at once. Whether they actually execute simultaneously is a separate question.

What is parallelism?

Parallelism is the simultaneous *execution* of computations on separate hardware resources — CPU cores, hardware threads, SIMD lanes, GPU cores, machines. Parallelism requires hardware; concurrency does not.

What is asynchronous programming?

Asynchronous programming is a style in which you *initiate* an operation and continue without synchronously waiting for it, then handle its completion later — through a callback, a future/promise, or await. Asynchronous code is concurrent (several operations are outstanding) but need not be parallel: Node.js and Python asyncio run many asynchronous tasks on one thread.

Concurrency versus parallelism Multiple tasks make progress during overlapping periods One server handles many clients

Multiple computations execute at the same instant on separate execution resources

Two CPU cores calculate different results Initiate an operation and continue without waiting synchronously for its completion

Await a network response without blocking the event loop Multiple threads execute within a process Worker threads process jobs

Separate processes execute independently Process pool for CPU-heavy work Non-blocking I/O

An operation returns without waiting for the resource to become ready Socket read reports "not ready"

Repeatedly dispatches ready callbacks/tasks and processes events Browser or server event loop

Asynchronous code can be concurrent without being parallel, and threads can run one after another on a single core while still being concurrent. Rob Pike's summary: *concurrency is about dealing with lots of things at once; parallelism is about doing lots of things at once.*

Synchronous versus asynchronous operations A synchronous call returns when its work is finished (read() returns the data). An asynchronous call returns a handle immediately (a promise, a future, a task) and the work completes later.

Blocking versus non-blocking operations A blocking operation suspends the calling thread until it can proceed (waiting for disk, network, a lock, a timer). A non-blocking operation never suspends: it does what it can immediately and reports "would block" otherwise. Asynchronous APIs are usually built on non-blocking operations plus a notification mechanism (Module 11).

Sequential versus concurrent execution Sequential execution finishes task A before starting task B. Concurrent execution interleaves or overlaps them. For I/O-bound work, overlapping waits is where the speed-up comes from (measured below).

CPU-bound versus I/O-bound workloads CPU-bound: time is spent computing (compression, image processing, simulation). Speed-ups come from *parallelism* — more cores.

I/O-bound: time is spent waiting for disks, networks, databases, users. Speed-ups come from *concurrency* — overlapping the waits — which needs no extra cores.

Latency versus throughput Latency is how long one operation takes; throughput is how many operations complete per unit time. Concurrency usually raises throughput; it can raise or lower individual latency (queuing adds waiting time — Module 22).

Responsiveness versus total execution time A GUI that moves work to a background thread may take *longer* in total but stays *responsive*: the user can still scroll and cancel. Responsiveness is often the main goal of concurrency in user-facing software.

Tasks, threads, processes and coroutines private address space heaviest (ms to create) Thread (OS / kernel thread) shares its process's memory, own stack ~10–100 µs to create, KiB–MiB of stack a runtime's scheduler (thread pool, event loop) cheap object, runs on some thread resumed by an event loop or scheduler at suspension points its frame/state object very cheap; thousands to millions

"Task" is used differently by different runtimes (.NET Task, Python asyncio.Task, RTOS tasks are threads); the course always says which one it means.

Cooperative versus preemptive scheduling Preemptive: the scheduler can interrupt a running thread at (almost) any instruction — OS threads on desktop and server OSes, most RTOS tasks.

Cooperative: a task runs until it voluntarily yields (await, yield, returning to the event loop). Simpler to reason about, but one task that never yields freezes everyone else.

Shared-memory versus message-passing concurrency In shared-memory concurrency, tasks communicate by reading and writing the same data, protected by synchronization (locks, atomics). In message-passing concurrency they send each other messages over channels or queues and share nothing mutable (Go channels, actors, Erlang). Most real systems mix both.

Deterministic versus nondeterministic behavior A deterministic program produces the same result and the same order of events for the same input. Concurrent programs are usually nondeterministic: the OS scheduler, CPU caches, I/O timing and other load change the interleaving on every run. *Correct* concurrent programs constrain that nondeterminism so that every possible interleaving yields an acceptable result.

Why concurrency is difficult The number of possible interleavings grows explosively with the number of steps.

Bugs may appear once in a million runs, only under load, or only on another CPU architecture.

Testing cannot enumerate all schedules; reasoning and tools (race detectors, model checkers) are needed.

Compilers and CPUs reorder memory operations (Module 7).

Failures compose badly: a timeout in one task must cancel others and clean up.

Typical real-world use cases Web and API servers, databases, browsers (UI thread + workers), mobile apps (main thread + background work), game engines (job systems), build systems (parallel compilation), data pipelines, scientific computing, operating systems, embedded controllers (ISRs + RTOS tasks), distributed services.

Course setup and code-reading conventions Every runnable example names its language, toolchain and platform (see the Course Overview).

Output blocks are real; when the output can vary, the course says *"one real run"* or shows several runs.

Pseudocode is always fenced as text and labelled *pseudocode*.

"Thread" means an OS thread unless stated otherwise.

Why It Matters in Real Systems

Choosing the wrong model wastes hardware or causes outages: adding threads to an I/O-bound Python service can help, while adding them to a CPU-bound one may not (the GIL — Module 17); running CPU-heavy code on a Node.js event loop freezes every client; one unsynchronised counter in a payment service loses money silently.

Key Terms

Overlapping lifetimes of independently progressing activities

Simultaneous execution on separate hardware Start an operation now, handle its completion later Blocking / non-blocking

Suspends the caller / returns immediately CPU-bound / I/O-bound Limited by computation / by waiting Latency / throughput

Time per operation / operations per time Preemptive / cooperative

Scheduler interrupts / task yields voluntarily One particular ordering of steps from concurrent tasks

Different runs can produce different orders or results

Syntax and API Overview

This is a concurrency concept, not a language construct. The same three-way comparison — sequential, threads, async — appears in every language:

pseudocode
sequential:  for each url: fetch(url)                        # waits one after another
threads:     start a thread per url; join all threads       # waits overlap; OS schedules
async:       start a task per url; await all tasks          # waits overlap; event loop schedules

Complete Code Example

Three simulated I/O waits of 0.3 s each, done three ways in CPython 3.13.1. Sleeping stands in for a network or disk wait.

import asyncio
import threading
import time


def blocking_io(name):
    time.sleep(0.3)                     # blocks this thread only
    return name


async def async_io(name):
    await asyncio.sleep(0.3)            # suspends this coroutine; the event loop runs others
    return name


def sequential():
    return [blocking_io(n) for n in "ABC"]


def with_threads():
    results = {}
    def run(n):
        results[n] = blocking_io(n)
    threads = [threading.Thread(target=run, args=(n,)) for n in "ABC"]
    for t in threads:
        t.start()
    for t in threads:
        t.join()                        # wait for every worker
    return [results[n] for n in "ABC"]


async def with_async():
    return await asyncio.gather(*(async_io(n) for n in "ABC"))


for label, fn in [("sequential", sequential), ("3 threads", with_threads), ("asyncio", lambda: asyncio.run(with_async()))]:
    start = time.perf_counter()
    result = fn()
    elapsed = time.perf_counter() - start
    print(f"{label:<11} {result}  {elapsed:.2f} s  (~{round(elapsed / 0.3)} x 0.3 s)")

Line-by-Line Explanation

blocking_io blocks the calling thread for 0.3 s; async_io *suspends* a coroutine at await, letting the event loop run other coroutines on the same thread.

sequential waits three times in a row.

with_threads starts three OS threads; while each one sleeps, the others can run. join() waits for all of them before reading results.

with_async starts three coroutines with asyncio.gather on one thread.

The last column rounds the elapsed time to multiples of 0.3 s, which is robust to timing noise.

Output

Output (one real run — timings vary slightly):

Both concurrent versions overlap the three waits, so they take about one wait instead of three. Neither used more than one CPU core meaningfully: this is concurrency for I/O-bound work, not parallelism.

sequential  ['A', 'B', 'C']  0.90 s  (~3 x 0.3 s)
3 threads   ['A', 'B', 'C']  0.30 s  (~1 x 0.3 s)
asyncio     ['A', 'B', 'C']  0.32 s  (~1 x 0.3 s)

Interleaving or State Trace

asyncio (one thread) A, B, C all waiting A, B, C all suspended at await A done, B waits all done → join returns event loop resumes A, B, C B done, C waits

Correctness and Safety Analysis

Each thread writes a different dictionary key, and the main thread reads only after join(), which waits for completion. That is safe here; in general, joining does not by itself make shared mutable state safe — every access must be synchronised (Modules 3–4).

Common Mistakes and Misconceptions

"Async makes code run in parallel." It overlaps waits; CPU-heavy code still runs one piece at a time on the event loop.

"More threads = faster." Only if there is waiting to overlap or idle cores to use.

"Concurrency and parallelism are synonyms." They are independent properties.

"If it worked once, it is correct." Concurrency bugs depend on timing.

Failure Modes and Debugging Steps

If the concurrent version is no faster: check whether the work is really I/O-bound, whether a lock or the GIL serialises it, and whether the "async" call is actually a hidden blocking call (time.sleep inside an async def blocks the whole loop).

Performance and Scalability Implications

Threads cost memory (stacks) and scheduling; coroutines cost little but need cooperative code; processes cost the most but give true isolation and parallelism. Module 22 quantifies these trade-offs.

Security Considerations

Concurrency introduces timing-dependent bugs that attackers can sometimes trigger deliberately (TOCTOU races, Module 25). Unbounded concurrency (one thread per request) is a denial-of-service risk.

Platform and Runtime Caveats

CPython's standard build has a global interpreter lock (GIL); threads overlap I/O well but do not run Python bytecode in parallel. Free-threaded builds of CPython 3.13+ exist as an experimental option (Module 17).

JavaScript runs each agent's code on one thread; parallelism needs workers.

Real-World Applications

A crawler fetching 1,000 pages (I/O-bound → async), a video transcoder (CPU-bound → processes or native threads), a GUI (responsiveness → background workers), a database (both).

Best Practices

Classify the workload first: CPU-bound or I/O-bound?

Prefer the simplest model that meets the need: sequential → async or a thread pool → processes.

Never assume an output order that the program does not enforce.

Exercises

E1 · Beginner. Classify each as CPU-bound or I/O-bound: resizing 10,000 images; downloading 10,000 images; a chat server; computing SHA-256 of a 10 GB file from an SSD.

Answer: CPU-bound; I/O-bound; I/O-bound (mostly waiting for messages); mostly CPU-bound (hashing), though a slow disk can make it I/O-bound — measure.

E2 · Intermediate. Give an example of concurrency without parallelism and of parallelism without concurrency in the program's design.

Answer: Concurrency without parallelism: Node.js handling many sockets on one thread. Parallelism without concurrent design: a single SIMD instruction adding eight numbers at once, or a CPU pipeline overlapping instructions of one sequential program.

E3 · Advanced challenge. Change the example to do CPU work (sum(i*i for i in range(3_000_000))) instead of sleeping, and compare sequential, threads and asyncio. Explain the result.

Answer: All three take about the same total time on the standard CPython build: the GIL lets only one thread execute Python bytecode at a time, and asyncio never runs coroutines in parallel. Use ProcessPoolExecutor (Module 16) or a free-threaded build for CPU parallelism.

Quiz

? A Node.js server handles 10,000 open connections on one thread. This is...
+ concurrency without parallelism :: Many activities progress in overlapping time on one thread.
- parallelism without concurrency :: One thread cannot execute simultaneously on several cores.
- neither :: Many connections are in progress at once.
- multiprocessing :: There is only one process.

? Which workload benefits most from asynchronous I/O on a single thread?
- Rendering a 3D scene :: CPU-bound work needs cores, not overlapped waiting.
+ Calling 50 slow web APIs and combining the results :: The time is waiting; async overlaps the waits.
- Sorting a large array in memory :: CPU-bound.
- Compressing a file :: CPU-bound.

? Which statements are true? (select all that apply)
+ Threads on a single core can still be concurrent. :: The OS interleaves them.
- Async/await always creates new threads. :: Most async runtimes run many tasks on few threads, often one.
+ Cooperative scheduling depends on tasks yielding. :: A task that never yields blocks others.
+ A correct concurrent program must be correct for every possible interleaving. :: Not just the ones you happened to observe.

Summary

Concurrency is about structure (overlapping progress), parallelism about execution (simultaneous hardware), asynchrony about not waiting synchronously. I/O-bound work gains from overlapping waits; CPU-bound work needs parallel hardware. Processes, threads, tasks and coroutines trade isolation for cost; preemptive and cooperative scheduling trade control for simplicity. Concurrency is hard because interleavings are many and nondeterministic.