Coroutines, Generators and Cooperative Scheduling

A normal function is a phone call that must finish before you hang up.

Learning Objectives

Distinguish functions from coroutines; know coroutine states, suspension points and resume semantics; yield values with generators and async generators.

Explain cooperative scheduling, cancellation points, and coroutines versus threads; stackful versus stackless coroutines; fibers, green threads and lightweight tasks.

Understand coroutine frames, lifetime and captured variables, re-entrancy and concurrent resumption, structured scopes, scheduling policies, leaks and debugging.

Compare coroutines in Python, JavaScript, C++20, Kotlin, Go (goroutines) and Rust, and their allocation overhead.

Plain-Language Explanation

A normal function is a phone call that must finish before you hang up. A coroutine is a conversation you can put on hold: it can pause in the middle, hand control back, and later continue exactly where it stopped with everything it knew. Generators pause to hand out one value at a time; async coroutines pause while waiting for I/O. A scheduler decides which paused coroutine continues next — and because coroutines pause only where they choose to, the scheduling is *cooperative*.

Precise Technical Explanation

Function versus coroutine A subroutine runs from entry to return in one activation. A coroutine can suspend and later resume, possibly many times, keeping its local state between suspensions (Conway, 1963).

Created (not started) → running → suspended → (resumed …) → finished (returned or raised) / closed. Python exposes them via inspect.getgeneratorstate/getcoroutinestate (shown below).

Only at explicit points — yield, await, co_await/co_yield, Kotlin suspend calls, Go's runtime scheduling points (channel operations, syscalls, function preamble checks; since Go 1.14 also asynchronous preemption).

Resuming continues after the suspension point, optionally delivering a value (gen.send(v), the result of await) or an exception (gen.throw(e), cancellation).

yield v suspends and hands v to the resumer; the resumer's next()/send() returns it.

Coroutines specialised for producing sequences lazily (Python, JavaScript function*, C# iterators, C++23 std::generator). Memory-efficient pipelines without threads.

Generators that may also await between yields (async def … yield, async function*, IAsyncEnumerable) — streaming data from asynchronous sources.

A scheduler resumes coroutines one at a time; each runs until its next suspension. No preemption means no data races between coroutines on one thread, but one coroutine that never yields blocks all.

Where a coroutine can observe cancellation: its suspension points (Python raises CancelledError at the await; Kotlin's suspending functions check the Job; Rust drops the future at an .await). CPU loops without suspension points cannot be cancelled until they yield (yield()/ensureActive() in Kotlin, await asyncio.sleep(0) in Python).

Coroutines versus threads kernel context switch small frame (hundreds of bytes – KiB) stack (KiB – MiB) cooperative (by runtime) preemptive (by OS) only with a multi-threaded runtime blocks the whole carrier thread blocks only that thread

Stackful versus stackless coroutine concepts Stackless coroutines (Python, JavaScript, C#, Rust, Kotlin, C++20) store only the coroutine's own frame; they can suspend only directly in the coroutine body — callers must themselves be coroutines (the "function colouring" of async).

Stackful coroutines (Go goroutines, Lua coroutines, Boost.Context fibers, Java virtual threads) have their own (growable) stack and can suspend from any depth of ordinary calls.

Fibers and green threads User-mode threads scheduled by a runtime rather than the OS: Windows fibers, early Java green threads, Ruby fibers, Go goroutines (M:N), Java 21 virtual threads.

Runtime-managed units cheap enough to create per request or per item (goroutines start with a 2–8 KiB stack; virtual threads a few hundred bytes plus stack chunks on the heap; a Python task is an object plus a coroutine frame).

Coroutine frame/state storage Where suspended locals live: heap-allocated frame objects (Python, C# state machines, C++20 coroutine frames — elidable by the optimiser), the future value itself (Rust, often on the caller's stack or in a Box), or growable stacks (Go).

Coroutine lifetime and captured variables A suspended coroutine keeps its locals and references alive. Captured references to objects that die (C++20 coroutines taking const std::string& parameters to temporaries!) dangle after the first suspension.

Re-entrancy and concurrent resumption A generator cannot be resumed while it is running (ValueError: generator already executing); in C++ resuming a coroutine handle from two threads concurrently is a data race. Each coroutine must have one resumer at a time.

Structured coroutine scopes Parent scopes that own child coroutines and wait for them (Module 13).

FIFO ready queues (asyncio), work-stealing multi-threaded schedulers (Tokio, Kotlin Dispatchers.Default, Go), dedicated threads (UI dispatchers), priority-based (rare).

Coroutine leaks and un-awaited coroutines Coroutines that are created but never driven (never awaited, never polled) or suspended forever (waiting on a channel nobody writes) hold memory and resources. Python warns on never-awaited coroutines; Go has no warning for blocked goroutines (use pprof/goleak).

Debugging suspended tasks asyncio.all_tasks() + task.print_stack(), Python aiomonitor; Go goroutine dumps; Kotlin coroutine debug mode (-Dkotlinx.coroutines.debug);.NET Parallel Stacks/Tasks windows; Java jcmd Thread.dump_to_file (includes virtual threads).

Coroutines in Python Generators (yield, send, throw, close), native coroutines (async def/await) driven by asyncio or Trio, async generators.

Coroutines in JavaScript Generators (function*, iterators), async functions (promises), async generators (for await). Each coroutine runs on the agent's single thread.

Coroutines in C++20 The language provides only the *machinery*: keywords co_await, co_yield, co_return and the promise_type/coroutine_handle protocol. There is no scheduler or task type in C++20 (C++23 adds std::generator; executors/std::execution arrive in C++26). Libraries (cppcoro, folly, Boost.Asio, libunifex) provide them. Example below.

Coroutines in Kotlin suspend functions compiled to continuation-passing state machines; builders launch/async; dispatchers choose threads; structured concurrency by default.

Goroutines in Go go f() starts a stackful lightweight thread multiplexed onto GOMAXPROCS OS threads by the runtime's work-stealing scheduler; blocking syscalls hand off the OS thread.

Rust async futures async fn returns a state-machine Future; nothing runs until an executor polls it; dropping a future cancels it; Pin keeps self-referential futures in place. Executors come from crates (Tokio, async-std, smol).

Coroutine performance and allocation overhead Creating and switching coroutines is far cheaper than threads (shown below: 100,000 goroutines), but each suspension in stackless designs may allocate a frame and each await costs a scheduler round-trip; very fine-grained async can be slower than plain loops.

Why It Matters in Real Systems

Coroutines let servers handle one logical "thread of control" per connection without one OS thread per connection; generators make data pipelines lazy and memory-bounded; understanding stackful vs stackless explains "async colouring" and why blocking inside coroutines is harmful.

Key Terms

Function that can suspend and resume Coroutine that yields a sequence of values Place where a coroutine may pause Stackless / stackful

Saves only its own frame / has its own stack Green thread / fiber Thread scheduled in user space Go's stackful lightweight thread Where cancellation can be observed

Diagram or Execution Timeline

flowchart LR
  C[created] -- first resume --> R[running]
  R -- yield / await --> S[suspended]
  S -- resume / send --> R
  R -- return / raise --> F[finished]
  S -- close / cancel --> F

Syntax and API Overview

def gen():            # generator: next(g), g.send(v), g.throw(e), g.close()
    x = yield 1
async def coro():     # native coroutine: driven by an event loop; await other awaitables
    await other()

Complete Code Example

Generator states, send, a cooperative round-robin scheduler built from generators, and the frame-size difference between a coroutine and a thread (CPython 3.13.1):

The C++20 coroutine machinery, made into a minimal generator by hand (G++ 11.2 -std=c++20 -fcoroutines; the standard library has no generator until C++23):

Goroutines are cheap enough to create by the hundred thousand (Go 1.23.4, one real run):

Kotlin coroutines for comparison (not run — no Kotlin toolchain on the build machine; kotlinx.coroutines 1.8):

import inspect
import sys
import threading


def averager():
    total, count = 0, 0
    average = None
    while True:
        value = yield average            # suspension point: hands out the average, receives the next value
        total += value
        count += 1
        average = total / count


g = averager()
print("state after creation:", inspect.getgeneratorstate(g))
next(g)                                   # prime: run to the first yield
print("state after priming:", inspect.getgeneratorstate(g))
print("send 10 ->", g.send(10), "| send 20 ->", g.send(20), "| send 60 ->", g.send(60))
g.close()
print("state after close:", inspect.getgeneratorstate(g))


def task(name, steps):
    for i in range(steps):
        print(f"  {name} step {i}")
        yield                             # cooperative: give the scheduler a turn


def round_robin(tasks):
    ready = list(tasks)
    while ready:
        t = ready.pop(0)
        try:
            next(t)                       # resume until its next yield
            ready.append(t)
        except StopIteration:
            pass                          # finished


print("round-robin cooperative scheduling:")
round_robin([task("A", 3
…

state after creation: GEN_CREATED
state after priming: GEN_SUSPENDED
send 10 -> 10.0 | send 20 -> 15.0 | send 60 -> 30.0
state after close: GEN_CLOSED
round-robin cooperative scheduling:
  A step 0
  B step 0
  C step 0
  A step 1
  B step 1
  A step 2
a suspended generator object: 176 bytes (+ frame); a new thread reserves a stack of the OS default (threading.stack_size() = 0 means 'platform default', typically 1-8 MiB)

#include <coroutine>
#include <iostream>
#include <optional>

template <typename T>
struct Generator {
    struct promise_type {                                    // the protocol the compiler requires
        std::optional<T> current;
        Generator get_return_object() { return Generator{std::coroutine_handle<promise_type>::from_promise(*this)}; }
        std::suspend_always initial_suspend() noexcept { return {}; }   // start suspended (lazy)
        std::suspend_always final_suspend() noexcept { return {}; }
        std::suspend_always yield_value(T v) { current = v; return {}; } // co_yield suspends here
        void return_void() {}
        void unhandled_exception() { throw; }
    };
    std::coroutine_handle<promise_type> h;
    explicit Generator(std::coroutine_handle<promise_type> handle) : h(handle) {}
    Generator(Generator&& o) noexcept : h(std::exchange(o.h, {})) {}
    ~Generator() { if (h) h.destroy(); }                     // the frame is freed by its owner
    std::optional<T> next() {
        if (!h || h.done()) return std::nullopt;
        h.resume();                                          // run to the next co_yield or the end
        if (h.done()) return 
…

created; nothing has run yet
first ten Fibonacci numbers: 0 1 1 2 3 5 8 13 21 34

package main

import (
	"fmt"
	"runtime"
	"sync"
	"time"
)

func main() {
	var before, after runtime.MemStats
	runtime.GC()
	runtime.ReadMemStats(&before)
	start := time.Now()

	const n = 100_000
	var wg sync.WaitGroup
	release := make(chan struct{})
	wg.Add(n)
	for i := 0; i < n; i++ {
		go func() {
			defer wg.Done()
			<-release // every goroutine parks here, suspended
		}()
	}
	runtime.ReadMemStats(&after)
	fmt.Printf("%d goroutines parked: runtime.NumGoroutine()=%d, created in ~%d ms\n",
		n, runtime.NumGoroutine(), time.Since(start).Milliseconds()/10*10)
	fmt.Printf("stack memory in use grew by ~%d KiB (~%d bytes per goroutine)\n",
		(after.StackInuse-before.StackInuse)/1024, (after.StackInuse-before.StackInuse)/n)
	close(release) // wake them all
	wg.Wait()
	fmt.Println("all finished; GOMAXPROCS =", runtime.GOMAXPROCS(0))
}

100000 goroutines parked: runtime.NumGoroutine()=100001, created in ~470 ms
stack memory in use grew by ~800000 KiB (~8192 bytes per goroutine)
all finished; GOMAXPROCS = 8

import kotlinx.coroutines.*

fun main() = runBlocking {                       // a scope that blocks main until children finish
    val jobs = List(100_000) { i ->
        launch { delay(100); if (i == 0) println("child 0 resumed on ${Thread.currentThread().name}") }
    }
    jobs.joinAll()
    println("100,000 coroutines completed")
}

Line-by-Line Explanation

averager keeps total/count between sends: its frame survives each suspension. It must be primed (next(g)) to reach the first yield.

The round-robin scheduler resumes each generator until its next yield and re-queues it — a complete cooperative scheduler in a dozen lines; the output interleaves A, B, C deterministically.

C++: promise_type tells the compiler how to create, suspend and finish the coroutine; initial_suspend returning suspend_always makes it lazy; co_yield a calls yield_value and suspends; next() resumes it. The frame (holding a, b, i) is heap-allocated and destroyed by ~Generator.

Go: 100,000 goroutines each block on a channel receive; the runtime parks them without OS threads; stack memory per goroutine is a few KiB.

Output

Shown under each program (the Go timing and memory figures are from one real run).

Interleaving or State Trace

Round-robin queue evolution:

Ready queue after

Correctness and Safety Analysis

Cooperative scheduling on one thread makes everything between two yields atomic with respect to other coroutines — the generator scheduler needs no locks. That guarantee disappears with multi-threaded coroutine runtimes (Go, Tokio multi-thread, Kotlin Default), where goroutines/coroutines run in parallel and shared state needs synchronization.

Common Mistakes and Misconceptions

Forgetting to prime a generator before send.

Holding references (C++ const& parameters, Rust borrows) across suspensions to objects that may die.

Doing blocking or CPU-heavy work inside coroutines on a shared loop.

Assuming goroutines are free: blocked goroutines leak memory forever.

Resuming the same coroutine from two threads.

Failure Modes and Debugging Steps

Generator "already executing": a generator resumed itself (re-entrancy).

Memory growth with many suspended tasks: list them (asyncio.all_tasks(), goroutine profile) and find what they wait for.

C++ coroutine crashes after a suspension: dangling reference in the frame — take parameters by value.

Performance and Scalability Implications

Coroutines and goroutines allow hundreds of thousands of concurrent activities; their per-switch cost is low, but allocation per frame and scheduler hops add up for tiny tasks. Stackful designs (Go) pay for stack growth and copying; stackless designs pay for frame allocation (often optimised away).

Security Considerations

Each suspended coroutine pins memory; attackers who can open many slow connections can exhaust memory in coroutine servers just as with threads — cap concurrent connections and use timeouts.

Platform and Runtime Caveats

GCC 10–11 require -fcoroutines; GCC 12+, Clang 14+ and MSVC 19.28+ enable C++20 coroutines with -std=c++20.

Python's threading.stack_size() returns 0 unless set; actual thread stack sizes are platform defaults.

Goroutine minimum stack size is a runtime detail (2 KiB in many versions, grows by copying).

Real-World Applications

Python data pipelines with generators; asyncio/Trio servers; Go microservices (one goroutine per request); Kotlin Android apps (coroutines on Dispatchers.Main/IO); C++ networking with Boost.Asio coroutines; game scripting with Lua coroutines.

Best Practices

Use generators for lazy sequences; coroutines for I/O concurrency.

Never block inside a coroutine on a shared loop; offload.

Take coroutine parameters by value (C++) and keep task references (Python).

Give every coroutine an owner scope (Module 13).

Monitor the number of live tasks/goroutines.

Exercises

E1 · Beginner. Why must averager be primed with next(g) before the first send(10)?

Answer: A new generator has not reached any yield, so there is nowhere to deliver the sent value; send with a non-None value on a just-created generator raises TypeError. Priming runs it to the first yield.

E2 · Intermediate. Explain why an async function cannot be called from a normal function and awaited there ("function colouring"), and why Go and Java virtual threads avoid this.

Answer: Stackless coroutines can only suspend in their own body; to suspend at an inner call, every caller up the chain must also be a coroutine that awaits. Stackful runtimes keep a separate stack per lightweight thread, so any ordinary function can block and the runtime simply parks the whole stack.

E3 · Advanced challenge. Extend the round-robin scheduler so a task can yield ("sleep", seconds) and is resumed only after that time.

Answer: Keep a heap of (wake_time, task); when a task yields a sleep request, push it with now + seconds instead of re-queuing; each loop iteration moves due tasks from the heap to the ready queue and, if the ready queue is empty, time.sleep until the earliest wake time — this is the core of an event loop's timer handling.

Quiz

? What is the key difference between stackless and stackful coroutines?
+ Stackless ones can suspend only in their own body; stackful ones can suspend from any call depth. :: Stackful coroutines have their own stack.
- Stackless coroutines cannot return values. :: Both can.
- Stackful coroutines are always faster. :: Each design has costs.
- Stackless coroutines run in parallel automatically. :: Parallelism depends on the runtime.

? In C++20, what does the standard library provide for coroutines?
+ Only the language machinery and basic awaitables; no scheduler or general task type. :: Libraries supply tasks and executors.
- A full async runtime with an event loop. :: Not in C++20.
- Green threads. :: C++20 coroutines are stackless.
- std::async built on coroutines. :: std::async predates and is unrelated to coroutines.

? Which are true? (select all that apply)
+ Code between two yields of a single-threaded cooperative scheduler runs without interleaving from other coroutines. :: No preemption.
+ A coroutine suspended forever still holds its frame and everything it references. :: A leak.
- Go goroutines are scheduled cooperatively only at explicit yield calls. :: The runtime also 
…

Summary

Coroutines suspend and resume while keeping state; generators yield values; async coroutines await I/O. Stackless coroutines (Python, JS, C#, Rust, Kotlin, C++20) colour functions; stackful ones (Go, virtual threads, fibers) do not. Cooperative scheduling removes interleaving between suspension points but requires every coroutine to yield; ownership, cancellation points and leak monitoring keep large coroutine systems healthy.