14.4 Quantization: Rounding a Continuous Value to a Discrete Level

Explain what quantization error is, compute its maximum magnitude relative to the LSB, and distinguish ideal quantization limits from real-world ADC accuracy limits.

Quantization is the step inside every ADC where a continuous, infinitely-precise analog value is forced into one of a finite set of allowed digital levels — and because the true analog value almost never lands exactly on one of those levels, quantization always introduces a small, unavoidable error. This is fundamentally different from noise or a hardware defect: quantization error exists even in a mathematically perfect ADC, simply because representing a continuous quantity with a finite number of bits necessarily discards some information. Understanding quantization error — its size, its source, and its relationship to resolution — is essential for predicting how faithfully a digitized signal represents its original analog form.

Quantization Levels and Quantization Error

An N-bit converter divides its full-scale input range into 2^N quantization levels, each separated by one LSB (LSB ≈ Vref/2^N, the same step-size idea introduced for DACs in Section 14.1). When an analog input value falls between two adjacent levels — which is true for almost every possible analog value — the converter must choose the nearest level and represent the input as if it were exactly that level. The difference between the true analog value and the chosen quantized level is the quantization error.

For a converter that rounds to the nearest level (rather than always truncating downward), the quantization error is bounded to at most ±½ LSB — the worst case occurs when the true value falls exactly halfway between two adjacent levels, in which case the rounding decision introduces an error of exactly half a step in either direction. A converter that always truncates (rounds down) instead has quantization error bounded between 0 and a full +1 LSB, which is a strictly worse (larger, one-sided) error than round-to-nearest quantization.

Because quantization error behaves somewhat like added noise spread across the signal's frequency content, it is often analyzed as quantization noise, with a well-known approximate relationship between resolution and the resulting signal-to-quantization-noise ratio: SNR(dB) ≈ 6.02N + 1.76, meaning each additional bit of resolution improves the theoretical signal-to-noise ratio by roughly 6 dB.

• Quantization level spacing = LSB ≈ Vref / 2^N. • Quantization error = true analog value − chosen quantized level. • Round-to-nearest quantization: maximum error = ±½ LSB. • Truncating quantization: maximum error = up to +1 LSB (worse, one-sided). • Approximate SNR from quantization alone: SNR(dB) ≈ 6.02N + 1.76 — about 6 dB better per added bit.

Why More Bits Isn't the Whole Story

Increasing bit resolution reduces the ideal quantization step and therefore the theoretical maximum quantization error, but this is only one of several error sources that determine a real ADC's actual accuracy. Offset error shifts the entire transfer curve up or down from ideal; gain error changes its slope so full-scale doesn't land exactly where expected; differential and integral nonlinearity (DNL/INL) mean the real code steps aren't perfectly even or perfectly straight; reference voltage noise and drift directly corrupt every quantization decision since the levels themselves are defined relative to Vref; and thermal/electrical noise in the analog signal path can dwarf the theoretical quantization error entirely.

This is why an ADC's real, achieved precision is better described by its effective number of bits (ENOB) — a measured quantity derived from the converter's actual noise and distortion performance — rather than by its nominal bit count alone. A designer chasing precision has to budget for all of these error sources together, not just pick the highest bit count available.

• Ideal quantization error shrinks as N increases — but this is only the theoretical floor. • Real accuracy also depends on: offset error, gain error, DNL/INL, reference quality, and analog-path noise. • ENOB (effective number of bits) reflects measured real-world performance, and can be lower than the nominal resolution.

Quantization error bounds

These formulas describe the ideal, best-case quantization behavior only — real ADC error also includes offset, gain, linearity, reference, and noise contributions.

Quantization step = LSB ≈ Vref / 2^N

Round-to-nearest quantization error: |error| ≤ LSB / 2

Approximate quantization SNR: SNR(dB) ≈ 6.02 × N + 1.76

Code Examples

Because the converter rounds each sample to the nearest of 256 levels spaced about 19.53 mV apart, the worst-case rounding error is half that spacing — about 9.77 mV — occurring whenever the true analog value falls exactly between two adjacent quantization levels. Expected result: ≈ 9.77 mV

Max error ≈ LSB/2 ≈ 9.77 mV for this 8-bit, 5 V-reference converter.

N = 8, Vref = 5 V
LSB ≈ 5 V / 256 ≈ 19.53 mV

Max quantization error (round-to-nearest) = LSB / 2
                                          ≈ 19.53 mV / 2
                                          ≈ 9.77 mV

Important Rules

Quantization step = LSB ≈ Vref / 2^N.

Round-to-nearest quantization error is bounded to ±LSB/2.

Truncating quantization has a strictly worse, one-sided error bound of up to +1 LSB.

Increasing bit resolution reduces ideal quantization step, but real ADC accuracy also depends on offset, gain, noise, linearity, and reference quality.

Where quantization error matters

Audio engineers hear quantization error as low-level 'quantization noise' in low-bit-depth recordings (which is why dither is intentionally added before reducing bit depth), and instrumentation engineers budget quantization error alongside reference and offset error when specifying how many bits a measurement system actually needs.

Believing that quantization error can be eliminated entirely by using enough bits.

Recognize that quantization error is fundamental to representing a continuous value with a finite number of discrete levels — it shrinks with more bits but never reaches zero, and other error sources may dominate first.

Increasing bit resolution reduces the ideal quantization step, but real ADC accuracy also depends on offset, gain, noise, linearity, reference quality, and other errors — quantization error is an inherent, non-eliminable characteristic of any finite-resolution converter, not a fixable defect.

For a round-to-nearest quantizer, what is the maximum possible quantization error, in terms of the LSB? A. ±1 LSB B. ±½ LSB C. ±2 LSB D. 0 — quantization error can always be avoided

B. ±½ LSB — When a converter rounds to the nearest quantization level, the worst case occurs when the true value sits exactly halfway between two levels, giving a maximum error of ±½ LSB in either direction.

An engineer upgrades from a 10-bit ADC to a 16-bit ADC to reduce measurement error, but sees only a small real-world improvement. What is the most likely explanation? A. 16-bit ADCs are always defective B. Bit count has no relationship to quantization error C. Increasing bit resolution reduces the ideal quantization step, but real accuracy also depends on offset, gain, noise, linearity, and reference quality — one of these other error sources may now dominate D. The formula LSB ≈ Vref/2^N only applies to DACs, not ADCs

C. Increasing bit resolution reduces the ideal quantization step, but real accuracy also depends on offset, gain, noise, linearity, and reference quality — one of these other error sources may now dominate — This is the spec's core tricky point for quantization: the ideal quantization error shrinks with more bits, but real ADC accuracy is also limited by offset error, gain error, nonlinearity, reference quality, and noise — any of which can become the dominant error source once quantization error itself becomes small enough.

A 10-bit ADC has Vref = 3.3 V. Compute the LSB size and the maximum quantization error assuming round-to-nearest quantization.

LSB ≈ 3.3 V / 1024 ≈ 3.22 mV. Maximum quantization error ≈ LSB/2 ≈ 1.61 mV.

LSB ≈ Vref/2^N with N=10; max error for round-to-nearest quantization is half the LSB.