Compiler Optimization

Once code is in IR form, a compiler can apply a wide range of transformations that preserve the program's observable behavior while making it run faster, use less memory, or both. Optimization is where a compiler earns its keep beyond simply "translating" code.

Compiler optimization is the process of automatically rewriting a program's intermediate representation into an equivalent but more efficient form — doing the same job, producing the same observable results, but running faster and/or using fewer resources than a naive, unoptimized translation would.

Writing perfectly efficient code by hand for every situation is often impractical or would sacrifice readability — optimizations let a programmer write clear, natural code while the compiler automatically eliminates a great deal of redundant or wasteful work behind the scenes.

Every production compiler applies optimizations by default at typical build settings (though usually far fewer, or none, in a debug build, to keep compilation fast and debugging straightforward) — the difference between an unoptimized and a heavily optimized build of the same program can easily be several times in execution speed.

Imagine handing a first, unpolished essay draft to a skilled editor. The editor doesn't change your point — they cut redundant sentences, merge repeated ideas, and reorganize for a smoother, faster read, while the essay still says exactly what you originally meant. Compiler optimization does the same to code: the program's meaning and results stay identical, but redundant, repeated, or wasteful operations get trimmed away.

Since 2 + 3 involves only compile-time-known constants, the compiler computes the result once during compilation instead of generating an actual addition instruction that would otherwise run every single time the program executes this line.

Constant folding evaluates constant expressions at compile time rather than at runtime (as shown above). Constant propagation goes further, tracking that a variable holds a known constant value and substituting that value wherever the variable is used, potentially enabling further folding.

Dead-code elimination removes code whose result is never used and has no observable effect (like an assignment to a variable that's never read afterward), and common subexpression elimination avoids recomputing an expression whose value the compiler can prove hasn't changed since it was last computed.

Loop optimizations are especially valuable since loop bodies often execute many times — loop-invariant code motion moves computations that don't change across iterations outside the loop entirely, loop unrolling duplicates a loop's body to reduce the relative overhead of the loop's own bookkeeping, and function inlining replaces a function call with a copy of the function's actual body, avoiding call overhead and often enabling further optimization at the call site.

Optimizations are typically controlled by compiler flags (like GCC/Clang's -O0 through -O3, or -Os for optimizing for size) — higher optimization levels generally produce faster code at the cost of longer compilation time and, sometimes, harder-to-debug generated code, which is exactly why debug builds typically disable most optimizations.

Assuming optimizations always make code strictly faster with no trade-offs — some optimizations (heavy inlining, loop unrolling) increase code size, which can hurt instruction-cache performance, and can occasionally make code slower in specific circumstances despite generally helping.

Debugging an optimized build and being confused when the source line the debugger reports doesn't match what actually appears to be executing — optimizations can reorder, merge, or eliminate code in ways that break the simple one-to-one mapping between source lines and executed instructions, which is exactly why debug builds usually disable optimization.

Writing code that relies on undefined behavior and happening to work at a low optimization level, then being surprised when a higher optimization level produces different, seemingly broken results — the compiler is legally allowed to assume undefined behavior never occurs and can optimize accordingly, a point developed in the earlier Memory Safety & Determinism fundamentals topic.

Manually micro-optimizing code the compiler would already have transformed identically, spending effort and hurting readability for no actual runtime benefit — profiling first, before hand-optimizing, avoids this.

Using a release build's higher optimization level for production/shipped code, while keeping a debug build with optimizations disabled (or minimal) during active development, for a straightforward source-to-execution correspondence when debugging.

Profiling code to find where time is actually spent before manually optimizing anything by hand, since compiler optimizations already handle a great deal of the "obvious" work automatically, and hand-tuning the wrong section wastes effort.

Using a tool like Compiler Explorer to compare the generated assembly for the same source snippet at different optimization levels, building direct intuition for what specific optimizations actually do to real code.

Strength reduction and peephole optimization Strength reduction replaces an expensive operation with a cheaper equivalent one where possible — for example, replacing a multiplication by a power of two with an equivalent, faster bit-shift. Peephole optimization looks at a small "window" of adjacent generated instructions and replaces obviously suboptimal short sequences with better equivalents — both are narrowly scoped, mechanical optimizations, but applied pervasively across a whole program they add up to meaningful gains.

Register allocation and instruction scheduling Register allocation decides which of a small, fixed number of physical CPU registers should hold which of a program's (potentially far more numerous) IR-level values at any given point, spilling values to memory when there aren't enough registers to go around — widely considered one of the hardest and most impactful problems in code generation, closely related to graph coloring in computational complexity terms. Instruction scheduling reorders generated instructions (without changing their meaning) to better match a specific CPU's actual pipeline behavior, hiding latency and avoiding stalls — both are covered further in the following Code Generation section.

Link-time optimization (LTO) and profile-guided optimization (PGO)

Ordinary optimization only sees one translation unit (roughly, one source file) at a time, missing opportunities that span multiple files — LTO defers key optimization decisions until link time, when the whole program's code is available together, enabling cross-file inlining and other whole-program optimizations. PGO instead uses data collected from actually running the program (which branches are taken most often, which functions are "hot") to guide optimization decisions more accurately than static analysis alone could, at the cost of needing a representative profiling run as part of the build process.

What's the difference between constant folding and constant propagation?

Constant folding evaluates a constant expression at compile time instead of at runtime. Constant propagation tracks that a variable holds a known constant value and substitutes that value at its uses, which can then expose further opportunities for constant folding elsewhere.

Why might a program behave differently (or reveal a latent bug) at a higher optimization level?

Often because the code relies on undefined behavior that happened to behave a particular way at a lower optimization level — compilers are permitted to assume undefined behavior never occurs and can transform code accordingly, so a higher optimization level can expose an incorrectness that was previously masked.

What does function inlining do, and what's the trade-off?

It replaces a function call with a direct copy of that function's body at the call site, eliminating call overhead and often enabling further optimizations across what used to be a call boundary. The trade-off is increased code size, which — especially with aggressive or repeated inlining — can hurt instruction-cache performance and, in extreme cases, actually slow the program down.

What is profile-guided optimization?

An optimization approach that uses data collected from actually running the program (which code paths are hot, which branches are usually taken) to make more accurate optimization decisions than static analysis of the source alone could achieve, at the cost of requiring a representative profiling run as an extra build step.

// as written:
int x = 2 + 3;

// after constant folding, the compiler emits code as if you'd written:
int x = 5;