Generators and Lazy Iteration: yield, yield from, and Memory-Efficient Processing

Understand how PHP generator functions produce values lazily with yield, how yield from delegates to nested iterables, how a generator can carry a final return value, and when reaching for a generator meaningfully reduces memory usage compared to building and returning a full array.

Most PHP functions that 'return a list' build the entire list in memory before handing it back — fine for a hundred rows, disastrous for ten million. A generator function looks almost identical to a normal function but behaves completely differently: instead of computing every value up front, it pauses at each yield and hands control back to the caller, producing values one at a time, on demand, using a small constant amount of memory regardless of how many values it ultimately produces. This lesson covers the mechanics — yield, yield from, keys, sending values back in, and return values — and builds the intuition for when lazy iteration is the right tool.

What a generator actually is: a function that returns an iterator

Any function containing at least one yield keyword automatically becomes a generator function — PHP does not require a special return type declaration to trigger this (though you can and should type-hint the return as Generator or a more specific iterable/Generator<...> in a docblock for tooling). Calling a generator function does not run its body at all; it immediately returns a Generator object, which implements PHP's Iterator interface. The function body only starts executing when you begin iterating the Generator — with a foreach, or by manually calling ->current(), ->next(), ->valid(), and ->rewind().

Each time execution reaches a yield statement, the generator produces that value to whoever is iterating it, and then suspends — its entire local state (variables, the current point of execution, loop counters) is preserved exactly as it was. When the consumer asks for the next value (the next foreach iteration), execution resumes exactly where it left off, right after the yield, and continues until it hits the next yield, a return, or the end of the function body. This suspend/resume mechanism is what lets a generator represent a conceptually infinite or very large sequence without ever materializing more than one value in memory at a time.

• A function with any yield in its body is automatically a generator function. • Calling it does not execute the body — it returns a Generator object immediately. • The body only runs (up to the next yield) as the Generator is iterated. • Local state is fully preserved across each suspend/resume cycle.

yield with and without keys, and sending values in

Plain yield $value; produces a value with an automatically incrementing integer key, just like appending to an array. yield $key => $value; lets you control the key explicitly, useful when the generator is standing in for an associative array. Both forms can appear inside any control structure — loops, conditionals, even recursive calls — because yield is just an expression that can appear anywhere a value is needed.

yield is also an expression, not just a statement, meaning it evaluates to whatever value the consumer sends back in via $generator->send($value). This turns a generator into a two-way communication channel: the consumer can push data into the generator's paused execution point, not just pull data out. This is a more advanced pattern (used in coroutine-style code and some Fiber-adjacent libraries) — most everyday generator usage only ever pulls values out with a plain foreach and never calls send().

• yield $value; — auto-incrementing integer keys, like array_push(). • yield $key => $value; — explicit keys, like an associative array. • $x = yield $value; — yield as an expression; $x receives whatever was passed to ->send().

yield from: delegating to another iterable

yield from $iterable; delegates iteration to another array, Traversable, or nested generator, yielding all of its values (and, for generators, its keys) as if they had been yielded directly by the outer generator — without loading them into memory as an intermediate array first. This is essential for composing generators: you can build a large lazy sequence out of smaller lazy pieces, and yield from preserves laziness all the way through, rather than collapsing everything into an eager array at the point of delegation.

A particularly powerful use of yield from is flattening nested or recursive structures — walking a tree, recursively descending directories, or chaining several finite generators into one logical stream — entirely lazily, so that even a very deep or very large structure is processed one item at a time rather than being fully expanded into a single in-memory array first.

• yield from [1, 2, 3]; — delegates to a plain array. • yield from anotherGenerator(); — delegates to a nested generator, preserving laziness. • Keys from a delegated generator are passed through as-is (not renumbered), which can cause key collisions across multiple yield from calls — use plain values with yield from if you need clean, sequential keys.

Generator return values

A generator function can also return a final value (in addition to any number of yielded values) once it finishes — this is distinct from the yielded values themselves and is not visible while iterating with foreach. After iteration has fully completed (the foreach loop has ended, or ->valid() returns false), calling $generator->getReturn() retrieves that final return value. This is a clean way to have a generator report a summary or result after streaming through a large amount of data — for example, streaming rows from a file while yielding each parsed row, and returning a final count or list of errors once the stream is exhausted.

Calling getReturn() before iteration is complete throws an exception, because the return value genuinely does not exist yet — it is only produced once the generator's body actually reaches its return statement (or falls off the end, in which case the return value is implicitly null).

Memory-efficient processing: why this matters in practice

The core trade-off generators make is time/structure for memory: a function that returns array must fully compute and hold every element before returning anything, so processing (say) a 5 GB CSV file into an array of parsed rows means holding a large multiple of 5 GB in memory at once. The equivalent generator function yields one parsed row at a time, so memory usage stays roughly constant — proportional to one row, not the whole file — regardless of whether the file has a thousand rows or a hundred million.

This matters anywhere you're processing a data source that is large, unbounded, or simply unknown in size ahead of time: reading large files line by line, paginating through an API and yielding each page's items as you go, streaming database query results with an unbuffered cursor, or producing an infinite sequence (like consecutive Fibonacci numbers) that could never be materialized into a finite array in the first place. The consumer's code (a foreach loop) looks essentially identical whether it's iterating an array or a generator, which means you can often refactor an eager, memory-heavy function into a lazy generator with minimal changes to the calling code.

• Eager (array): must fully compute and hold every element in memory before returning anything. • Lazy (generator): computes and holds roughly one element at a time, regardless of total sequence length. • Ideal for: large file processing, API pagination, database cursors, and conceptually infinite sequences. • Calling code (foreach) is nearly identical whether consuming an array or a generator.

Generator function shape

Calling readLinesLazily() only creates the Generator — nothing runs until the foreach starts pulling values. The file handle stays open across suspensions and is safely closed in the finally block once iteration ends. getReturn() is only valid to call after the foreach loop has fully finished.

function readLinesLazily(string $path): Generator
{
    $handle = fopen($path, 'r');
    if ($handle === false) {
        throw new \RuntimeException("Cannot open {$path}");
    }

    $lineNumber = 0;
    $errorCount = 0;

    try {
        while (($line = fgets($handle)) !== false) {
            $lineNumber++;
            if (trim($line) === '') {
                $errorCount++;
                continue;
            }
            yield $lineNumber => rtrim($line, "\n");
        }
    } finally {
        fclose($handle);
    }

    return $errorCount; // retrievable via ->getReturn() after iteration completes
}

$gen = readLinesLazily('data.txt'); // body has NOT run yet
foreach ($gen as $lineNumber => $text) {
    echo "{$lineNumber}: {$text}" . PHP_EOL;
}
echo 'Skipped blank lines: ' . $gen->getReturn() . PHP_EOL;

Code Examples

eagerRange() must allocate and hold all one million integers in an array before it can return anything. lazyRange() produces exactly one integer at a time as the foreach pulls it, so peak memory use is essentially constant regardless of the range's size — the difference becomes dramatic as the range grows into the tens or hundreds of millions. Expected result: 500000500000 for ($i = $start; $i <= $end; $i++) { yield $i; }

yield from delegates directly to evens() and then odds(), streaming their values through interleavedReport() without ever materializing either sub-sequence into an array — the outer generator is just as lazy as its inner ones. Once both delegated generators are exhausted, interleavedReport()'s own return 'done'; becomes retrievable via getReturn(). Expected result: 0 2 4 6 1 3 5 done yield from evens($limit); yield from odds($limit);

The generator streams each successfully parsed row to the caller as it goes, while accumulating a running validity count and an error list internally. Only after the foreach has fully drained the generator does getReturn() become available, exposing the summary that was built up across the whole streaming pass. Expected result: Ada unknown valid=2 errors=row 1 missing id return ['valid' => $validCount, 'errors' => $errors];

function eagerRange(int $start, int $end): array
{
    $result = [];
    for ($i = $start; $i <= $end; $i++) {
        $result[] = $i;
    }
    return $result; // entire array held in memory at once
}

function lazyRange(int $start, int $end): Generator
{
    for ($i = $start; $i <= $end; $i++) {
        yield $i; // one value in memory at a time
    }
}

$eager = eagerRange(1, 1_000_000);   // allocates ~1,000,000 ints immediately
$lazy  = lazyRange(1, 1_000_000);    // allocates almost nothing yet

$sum = 0;
foreach ($lazy as $n) {
    $sum += $n; // values produced and discarded one at a time
}
echo $sum . PHP_EOL;

function evens(int $limit): Generator
{
    for ($i = 0; $i <= $limit; $i += 2) {
        yield $i;
    }
}

function odds(int $limit): Generator
{
    for ($i = 1; $i <= $limit; $i += 2) {
        yield $i;
    }
}

function interleavedReport(int $limit): Generator
{
    yield from evens($limit);
    yield from odds($limit);
    return 'done';
}

$report = interleavedReport(6);
foreach ($report as $n) {
    echo $n . ' ';
}
echo PHP_EOL . $report->getReturn();

function parseRows(array $rawRows): Generator
{
    $validCount = 0;
    $errors = [];

    foreach ($rawRows as $i => $row) {
        if (!isset($row['id'])) {
            $errors[] = "row {$i} missing id";
            continue;
        }
        $validCount++;
        yield ['id' => $row['id'], 'name' => $row['name'] ?? 'unknown'];
    }

    return ['valid' => $validCount, 'errors' => $errors];
}

$rows = [['id' => 1, 'name' => 'Ada'], ['name' => 'no id'], ['id' => 3]];
$gen = parseRows($rows);

foreach ($gen as $parsed) {
    echo $parsed['name'] . PHP_EOL;
}

$summary = $gen->getReturn();
echo "valid={$summary['valid']} errors=" . implode(', ', $summary['errors']) . PHP_EOL;

Suspend/resume across a foreach loop

Each foreach iteration resumes exactly where the generator last suspended, runs to the next yield (or the end), and the cycle repeats — only one value's worth of work happens between resumes.

foreach ($gen as $v) { use($v); }

  call gen()  ->  Generator object returned, body NOT run
  foreach starts ->  body runs until first `yield` -> $v = value1, body SUSPENDED
  use($v1)
  next iter    ->  body RESUMES right after yield -> runs until next `yield` -> $v = value2, SUSPENDED
  use($v2)
  ...
  body reaches `return` or end -> generator exhausted -> getReturn() now available

Important Rules

A function containing any yield automatically becomes a generator — calling it returns a Generator without running its body.

Use yield from to delegate to nested arrays/iterables/generators while preserving laziness all the way through.

A generator's return value is separate from its yielded values — retrieve it with getReturn() only after iteration is fully complete.

Prefer a generator over building and returning a full array whenever the sequence is large, unbounded, or streamed from an external source (file, API, DB cursor).

Remember a Generator is a single-use, forward-only Iterator — you cannot rewind() it once iteration has started past its first element.

Where this shows up

ORM libraries expose 'lazy collection' or 'cursor' modes built on generators so you can foreach over millions of database rows without loading them all into memory; CSV/log-processing tools and ETL pipelines use generators chained with yield from to stream-transform data through multiple stages; and HTTP client libraries that paginate through an API commonly return a generator that lazily fetches the next page only when the caller's foreach actually asks for more items.

$gen = makeGenerator(); foreach ($gen as $v) {} foreach ($gen as $v) {} // second loop silently does nothing

$values = iterator_to_array(makeGenerator()); // if you truly need to iterate more than once, materialize it explicitly first

A Generator is forward-only and single-use: once fully iterated, a second foreach over the same instance produces no values (and PHP throws if you try to rewind() past the start). If you need to iterate the same data more than once, either call the generator function again for a fresh Generator, or explicitly convert it to an array with iterator_to_array() when you know it's small enough to hold in memory.

What happens when you call a function whose body contains a yield statement? A. The function body runs immediately and yield behaves like an early return B. PHP throws a syntax error unless the return type is declared Generator C. The function immediately returns a Generator object, and the body does not run until the Generator is iterated D. The function runs to completion and returns an array of all yielded values

C. The function immediately returns a Generator object, and the body does not run until the Generator is iterated — Calling a generator function returns a Generator object right away without executing any of the function body; execution only proceeds, up to the next yield, as the Generator is actually iterated.

What does yield from $otherGenerator; do? A. It converts $otherGenerator into an array before yielding its values B. It delegates iteration to $otherGenerator, lazily yielding its values without materializing them into an array first C. It is a syntax error — yield from only works with plain arrays D. It immediately exhausts $otherGenerator and discards its values

B. It delegates iteration to $otherGenerator, lazily yielding its values without materializing them into an array first — yield from delegates to another iterable (array, Traversable, or nested generator), passing its values (and keys, for generators) through lazily, preserving the outer generator's laziness rather than collapsing the delegated source into an eager array.

When is a generator's return value (set via a return statement inside it) available through getReturn()? A. At any time, even before iteration starts B. Only after iteration over the generator has fully completed C. Only if no values were ever yielded D. getReturn() does not exist in PHP; generators cannot have return values

B. Only after iteration over the generator has fully completed — getReturn() is only valid once the generator has been fully iterated (or otherwise reached its return/end) — calling it earlier throws, because the return value genuinely has not been produced yet.

Why would you choose a generator over a function that returns a full array when processing a very large file? A. Generators are always faster in raw CPU terms than array-returning functions B. Generators produce one item at a time, keeping memory usage roughly constant instead of holding every parsed item in memory at once C. Arrays cannot hold more than 10,000 elements in PHP D. Generators automatically run in a separate thread

B. Generators produce one item at a time, keeping memory usage roughly constant instead of holding every parsed item in memory at once — The core benefit is memory, not raw speed: an array-returning function must hold every element in memory before returning, while a generator yields one item at a time, keeping peak memory roughly constant regardless of how large the overall sequence is.

Write a generator function fibonacci(int $count): Generator that yields the first $count Fibonacci numbers (starting 0, 1, 1, 2, 3, 5, ...) one at a time, and returns the total sum of all yielded numbers as its final return value (retrievable via getReturn()).

Keep two running variables for the current and next Fibonacci numbers, yield the current one each iteration, and accumulate a running sum alongside them to return at the end.