How a Program Becomes Executable
A compiler driver such as gcc looks like one tool, but it is a production line.
Learning Objectives
Follow a C file through preprocessing, compilation (with IR and optimization), assembly, object files, linking and loading.
Explain sections, symbols, relocations, static and dynamic linking, and debug information.
Compare ELF, PE and Mach-O at a conceptual level.
Distinguish compilers, interpreters, bytecode virtual machines, JIT and AOT compilation.
Understand build systems, debug/release configurations, reproducible builds, cross-compilation and the embedded firmware flow.
Inspect every intermediate output with gcc -E/-S/-c, nm, objdump, readelf and ldd.
Prerequisites
Module 0. Being able to compile a C program from a terminal helps.
Plain-Language Explanation
A compiler driver such as gcc looks like one tool, but it is a production line. The preprocessor pastes in header files and expands macros. The compiler checks the code, translates it into an internal form, improves it and writes assembly. The assembler turns assembly into machine code inside an object file — a half-finished piece with holes where addresses of other functions should go. The linker joins object files and libraries and fills those holes, producing an executable. When you run it, the operating system's loader places it in memory and starts it.
Precise Technical Explanation
Source code and source files A C program is a set of *translation units*: each .c file plus every header it includes. Each unit is compiled independently; only the linker sees the whole program. Other languages differ — Java compiles classes, Rust compiles whole crates, Python compiles modules at import time.
Preprocessing and macro expansion The preprocessor handles every line starting with #: #include copies a file in, #define creates macros, #if/#ifdef remove code. Its output (gcc -E) is pure C with no directives. A five-line program that includes <stdio.h> becomes hundreds of lines because the header's declarations are pasted in.
Compilation and semantic analysis The compiler *lexes* the text into tokens, *parses* tokens into a syntax tree, and performs *semantic analysis*: name lookup, type checking, implicit conversions, and diagnostics such as "incompatible pointer type".
Intermediate representation (IR) The checked program is lowered to an IR — GCC's GIMPLE and RTL, LLVM IR for Clang/Rust/Swift. IR is simpler than source and independent of both the source language and the target CPU, which lets one optimizer serve many front ends and back ends.
Optimization passes transform IR without changing *observable behavior* as the language defines it: constant folding, dead-code elimination, inlining, loop transformations, register allocation, vectorisation. -O0 does almost none (best for debugging); -O2 is the usual release level; -Os optimises for size. Optimizers may delete variables, merge functions and reorder operations — which is why a debugger sometimes says *"value optimized out"*.
The back end selects target instructions, allocates registers and emits assembly — a text form of machine instructions (gcc -S). Assembly is architecture-specific: the same function produces different text for x86-64, AArch64 and RISC-V.
Assembling object files The assembler encodes instructions into bytes and writes an object file (.o on Unix, .obj on Windows). An object file contains machine code, data, a symbol table and relocation records — but not final addresses.
Object-file sections and symbols Common sections:
read-only data such as string literals initialised writable globals and statics zero-initialised globals and statics — takes no space in the file .symtab / .strtab symbol table and its names relocations to apply to .text DWARF debug information
A symbol is a named location: a *defined* symbol (T main — in text) or an *undefined* symbol (U puts — needed from elsewhere).
The linker combines object files and copies the needed members out of static libraries (.a, archives of .o files) into one executable. The result does not need the library at run time but is larger and must be relinked to pick up library fixes.
With shared libraries (.so, .dll, .dylib) the executable records only the library's name. At start-up the dynamic loader (ld-linux-x86-64.so.2 on Linux) maps the library and resolves the references. Many processes share one physical copy of the library's code.
Relocation and symbol resolution *Symbol resolution* matches every undefined symbol to exactly one definition (or reports "undefined reference" / "multiple definition"). *Relocation* patches the placeholder bytes in the code with the final addresses or offsets, as described by relocation records such as R_X86_64_PLT32 puts.
Executable formats: ELF, PE and Mach-O Linux, BSD, most embedded toolchains Code / data units sections grouped into *segments* *segments* containing *sections* Windows loader in ntdll
All three describe the same ideas: what bytes to map where, with which permissions, what to import and where to start.
Debug and symbol information -g adds DWARF (ELF/Mach-O) or PDB (Windows) data that maps machine addresses back to file names, line numbers, variables and types. It does not change the generated code. strip removes symbols and debug information to shrink release binaries; keep an unstripped copy to symbolise crash reports.
Build systems and dependency graphs Large programs are built by tools (Make, Ninja, CMake, MSBuild, Gradle, Cargo) that model a dependency graph: each output depends on inputs, and only out-of-date outputs are rebuilt.
Changing util.h rebuilds both object files; changing main.c rebuilds only main.o and relinks.
Compiler versus interpreter A compiler translates the whole program ahead of time into another form. An interpreter reads a program (or its bytecode) and performs each operation itself. Real systems mix both: CPython compiles source to bytecode, then interprets the bytecode.
A just-in-time compiler translates bytecode or IR into machine code *while the program runs*, focusing on hot code and using run-time information (actual types, branch frequencies). The JVM (HotSpot), .NET (RyuJIT) and JavaScript engines (V8, SpiderMonkey) all JIT-compile.
Bytecode and virtual machines Bytecode is a compact instruction set for a virtual machine rather than a real CPU: JVM bytecode in .class files, CIL in.NET assemblies, CPython bytecode in .pyc files. The VM provides memory management (garbage collection), a uniform instruction set and safety checks.
Ahead-of-time (AOT) compilation AOT compiles to native code before execution — the C/C++/Rust/Go default, and an option for managed languages (.NET Native AOT, GraalVM Native Image). AOT gives fast start-up and no JIT warm-up but loses some run-time-informed optimizations.
Build configurations: debug and release -O2 / -O3 / -Os often separate or stripped often disabled (-DNDEBUG) sanitizers, extra checks LTO, hardening flags
Bugs that appear only in release builds usually mean the program has undefined behavior that the optimizer exploited.
Reproducible builds and build artifacts A build is reproducible if the same source, tools and options produce bit-identical output. Embedded timestamps (__DATE__), absolute paths and random ordering break it. Reproducibility lets anyone verify that a binary matches its source — an important supply-chain defence.
Cross-compilation and target architecture A cross-compiler runs on one machine (the *host*, e.g. x86-64 Linux) and produces code for another (the *target*, e.g. Cortex-M4). Its name usually encodes the target triple: arm-none-eabi-gcc, aarch64-linux-gnu-gcc, riscv64-unknown-elf-gcc.
Embedded firmware build flow The linker script decides which sections go into flash and which into RAM (Module 16).
gcc -E (preprocessed), gcc -S (assembly), gcc -c (object), nm (symbols), objdump -d (disassembly), readelf -h/-S/-r/-d (headers, sections, relocations, dynamic section), ldd (shared-library dependencies), size (section sizes). Windows equivalents: dumpbin /headers /symbols /disasm and link /map; macOS: otool and nm.
flowchart LR
A[main.c] --> AO[main.o]
H[util.h] --> AO
H --> UO[util.o]
U[util.c] --> UO
AO --> E[app]
UO --> Eflowchart LR
C[C sources + startup.s] --> O[object files]
O --> L[linker + linker script]
L --> ELF[firmware.elf with symbols]
ELF --> BIN[objcopy to .bin or .hex]
BIN --> F[flash programmer writes MCU flash]
ELF --> MAP[map file: sizes and addresses]Why It Matters in Real Systems
"Undefined reference", "multiple definition", "cannot open shared object file" and "works in debug, crashes in release" are pipeline problems. Understanding the stages lets you fix them in minutes, and lets you check what the compiler really did with performance-critical code.
Key Terms
One source file plus all headers it includes Intermediate representation used by the optimizer
Machine code plus symbols and relocations, not yet linked Named function or variable, defined or undefined
Instruction to patch an address once it is known Archive of object files copied in at link time
Library mapped at load/run time and shared between processes
Program that maps shared libraries and resolves symbols at run time Compile while running / before running
Compiler whose target differs from its host
Syntax
On Windows with MSVC the corresponding commands are cl /P (preprocess), cl /FA (assembly listing), cl /c (object) and link. A compiler driver runs all stages automatically when you just write gcc main.c -o main.
gcc -E main.c -o main.i # preprocess only
gcc -S main.c -o main.s # compile to assembly
gcc -c main.c -o main.o # assemble to an object file
gcc main.o -o main # link
./main # run (Unix-like systems)Complete Runnable Example
The program (GCC 11.2, x86-64 Linux):
Every stage, inspected with the real tools (bash in WSL, GNU binutils 2.38):
#include <stdio.h>
int main(void)
{
puts("Program execution starts here.");
return 0;
}gcc -E m02_hello.c -o stage.i
echo "lines before / after preprocessing:"; wc -l < m02_hello.c; wc -l < stage.i
gcc -S -O2 -fno-asynchronous-unwind-tables m02_hello.c -o stage.s
echo "--- assembly (-O2): the string and main ---"; grep -A1 '^\.LC0:' stage.s; sed -n '/^main:/,/ret/p' stage.s
gcc -c m02_hello.c -o stage.o
echo "--- symbols in the object file ---"; nm stage.o
echo "--- relocations the linker must apply ---"; readelf -r stage.o | sed -n '/\.rela\.text/,/^$/p' | grep R_X86 | awk '{print $3, $5}'
gcc stage.o -o stage
echo "--- file types ---"; readelf -h stage.o | grep Type; readelf -h stage | grep Type
echo "--- shared libraries needed ---"; readelf -d stage | grep NEEDED
echo "--- run ---"; ./stageLine-by-Line Explanation
wc -l shows how much text #include <stdio.h> pastes in.
endbr64 marks a valid indirect-branch target for Intel CET (control-flow enforcement); Ubuntu's GCC adds it by default.
In the -O2 assembly, main loads the address of the string (.LC0) into %rdi — the first-argument register in the System V x86-64 ABI — and calls puts@PLT, a stub that jumps into the C library.
nm stage.o lists T main (defined in text) and U puts (undefined — the linker must find it).
The relocations say: "patch a PC-relative reference to .rodata" (the string) and "patch a call to puts through the PLT".
REL (relocatable object) becomes DYN — Ubuntu builds position-independent executables by default, which load at a random base (ASLR).
NEEDED libc.so.6 is all the executable records about the C library; the dynamic loader does the rest at start-up.
Expected Output
lines before / after preprocessing:
7
745
--- assembly (-O2): the string and main ---
.LC0:
.string "Program execution starts here."
main:
endbr64
subq $8, %rsp
leaq .LC0(%rip), %rdi
call puts@PLT
xorl %eax, %eax
addq $8, %rsp
ret
--- symbols in the object file ---
0000000000000000 T main
U puts
--- relocations the linker must apply ---
R_X86_64_PC32 .rodata
R_X86_64_PLT32 puts
--- file types ---
Type: REL (Relocatable file)
Type: DYN (Position-Independent Executable file)
--- shared libraries needed ---
0x0000000000000001 (NEEDED) Shared library: [libc.so.6]
--- run ---
Program execution starts here.Step-by-Step Execution Trace
Preprocess: #include <stdio.h> → hundreds of declarations, including int puts(const char *);.
Compile: the call is type-checked against that declaration and lowered to IR, optimised, and emitted as assembly.
Assemble: machine code is written to .text, the string to .rodata; the call site holds a placeholder and a relocation.
Link: the linker lays out sections, resolves puts to the shared C library through the PLT, and writes an ELF executable with an entry point (_start).
Load and run: the kernel maps the executable and ld-linux, the loader maps libc.so.6, _start calls __libc_start_main, which calls main.
Interactive Visualization
Click each stage of the pipeline above to see its input, output, purpose and typical errors, and switch between the ahead-of-time, interpreted and JIT paths.
Common Mistakes and Failure Modes
fatal error: foo.h: No such file or directory missing -I include path implicit declaration of function missing #include or prototype undefined reference to 'foo' object file or library (-lfoo) not linked, or name mismatch (C vs C++ mangling) multiple definition of 'x' a variable *defined* in a header included by several .c files
error while loading shared libraries: libfoo.so library not in the loader search path
Also: putting -lm *before* the object files with GNU ld (libraries are searched in command-line order).
Corrected / Safe Pattern
Link libraries after the objects that use them: gcc main.o util.o -lm -o app.
/* config.h — declare in the header ... */
extern int verbose_level;
/* config.c — ... define in exactly one source file */
int verbose_level = 1;Debugging and Diagnostic Techniques
Add -v to see every sub-command the driver runs; -save-temps keeps all intermediate files.
nm -C shows demangled C++ names; nm -u only undefined symbols.
objdump -d -S interleaves source and disassembly (needs -g).
readelf -d lists NEEDED libraries and RUNPATH; ldd shows where each one resolves.
Compiler Explorer (godbolt.org) shows assembly for many compilers and targets side by side.
Performance Implications
Optimization level dominates: -O2 code is commonly several times faster than -O0. Link-time optimization (-flto) lets the optimizer inline across files. Dynamic linking adds a small cost per first call (lazy binding) and per call through the PLT; static linking removes it at the cost of size.
Security and Safety Implications
Hardened builds add -fstack-protector-strong, -D_FORTIFY_SOURCE=2, PIE, full RELRO (-Wl,-z,relro,-z,now) and non-executable stacks. Library search-path hijacking (a malicious .so/.dll placed where the loader looks first) is a real attack; ship with fixed, trusted search paths. Reproducible builds let third parties verify binaries.
Platform and Language Caveats
PIE is the default on Ubuntu, Fedora and Debian; some distributions and most embedded toolchains produce fixed-address executables.
MSVC, Clang and GCC use different flags; the concepts are identical.
Java, C# and Python have no separate link step: classes and modules are loaded and resolved at run time.
The assembly shown is x86-64 AT&T syntax from GCC 11; another compiler, version or target produces different text.
Real-World Application
Continuous-integration systems run these stages for every commit, cache object files, and archive debug symbols so production crashes can be symbolised. Firmware teams read map files to keep code inside flash.
Best Practices Checklist
[ ] Build with -Wall -Wextra (and -Werror in CI).
[ ] Separate debug and release configurations; test both.
[ ] Keep unstripped binaries or separate debug symbols for every release.
[ ] Declare in headers, define in exactly one source file.
[ ] Pin toolchain versions and aim for reproducible builds.
Exercises
E1 · Beginner. Which stage reports each error: (a) a missing semicolon, (b) undefined reference to 'sqrt', (c) stdio.h: No such file?
Answer: (a) compilation (parser), (b) linking — add -lm after the objects, (c) preprocessing.
E2 · Intermediate. Why does .bss take no space in the executable file, and what makes it zero at run time?
Answer: The file only records its size; the loader maps anonymous zero-filled pages (or start-up code clears it on bare metal).
E3 · Advanced challenge. Compile the example with -O0 instead of -O2 and compare the assembly. Which instructions disappear at -O2, and why is printf("Hello\n") often compiled into a call to puts?
Answer: At -O0 GCC sets up a frame pointer (push %rbp; mov %rsp,%rbp) and loads/stores through the stack; -O2 removes the frame and keeps values in registers. printf of a constant string ending in \n with no format directives has the same effect as puts of the string without the newline, so GCC substitutes the cheaper call.
Quiz
? Which tool replaces #include lines with file contents?
+ The preprocessor :: It handles every # directive before compilation proper.
- The linker :: The linker never sees source code.
- The assembler :: The assembler reads assembly, which has no #include.
- The loader :: The loader works on executables at run time.
? An object file lists "U puts". What does that mean?
- puts is defined in this file's text section. :: That would be "T puts".
+ puts is used here but defined elsewhere; the linker must resolve it. :: U means undefined.
- puts is unused and will be removed. :: Unused symbols are not marked U.
- puts is a local static function. :: Local functions appear as "t".
? Which systems JIT-compile code at run time? (select all that apply)
+ The JVM (HotSpot) :: Hot methods are compiled to native code.
+ V8 for JavaScript :: V8 tiers from an interpreter to optimising compilers.
- A C program built with gcc -O2 :: That is ahead-of-time compiled.
+ .NET's CLR :: RyuJIT compiles CIL methods on first call (with tiered compilation).
? Why do debug and release builds sometimes behave differently?
- Debug builds use a different language standard. :: The standard is the same unless y
…Summary
A compiler driver runs preprocessing, compilation (front end, IR, optimizer, back end), assembly and linking. Object files carry code, data, symbols and relocations; the linker resolves symbols and applies relocations; the loader and dynamic loader map the result and shared libraries into a process. Interpreters, bytecode VMs, JIT and AOT are different points on the same spectrum.
References and Further Reading
John R. Levine, *Linkers and Loaders*.
*System V Application Binary Interface, AMD64 Architecture Processor Supplement*.
GCC manual: "Overall Options" and "Optimize Options".
Ian Lance Taylor, "Linkers" blog series (20 parts).