struct
Packing and unpacking Python values to and from raw fixed-layout binary data — the tool for reading binary file formats, network protocols, and C-compatible structures.
Most of the time, Python data interchange means text formats like JSON or CSV. But binary file formats, network protocol headers, and interfacing with C libraries or hardware all speak in fixed-width binary layouts — bytes with a precise size, order, and meaning, no delimiters or parsing needed. The struct module converts between Python values and these packed binary representations using a compact format-string mini-language.
What Is struct and Why It Matters
struct converts between Python values (ints, floats, booleans, fixed-length byte strings) and bytes objects representing them in a specific, fixed binary layout — the same kind of layout a C struct, a network protocol header, or a binary file format uses. You need it whenever you're reading or writing binary data with a known, fixed structure: image file headers, network packets, sensor data from hardware, or any file format documented in terms of "bytes 0-3 are a big-endian uint32, bytes 4-7 are a float32", and so on.
import struct
# Pack two ints and a float into 12 bytes of binary data
packed = struct.pack("iif", 42, -7, 3.14)
print(packed) # b'*\x00\x00\x00\xf9\xff\xff\xff\xc3\xf5H@'
print(len(packed)) # 12 bytes: 4 + 4 + 4
# Unpack it back
values = struct.unpack("iif", packed)
print(values) # (42, -7, 3.140000104904175) — float32 loses some precisionFormat Strings: The Core Syntax
A format string is a sequence of characters, each representing one C-compatible type, describing the exact layout of the packed bytes. A leading count before a character repeats that type (e.g. "4i" means four ints back-to-back, equivalent to "iiii").
Format char C type Python type Standard size (bytes)
----------- --------------- ------------- ----------------------
x pad byte (none) 1
c char bytes (len 1) 1
b / B signed/unsigned char int 1
h / H short / unsigned short int 2
i / I int / unsigned int int 4
l / L long / unsigned long int 4
q / Q long long / unsigned long long int 8
f float float 4
d double float 8
s char[] (string) bytes (count-dependent)
? _Bool bool 1import struct
# "4s" means a 4-byte fixed-length byte string, not four separate chars
packed = struct.pack("4sI", b"PNG\x00", 8)
print(packed)
print(struct.unpack("4sI", packed)) # (b'PNG\x00', 8)Endianness and Alignment
An optional first character in the format string selects byte order and alignment. @ (default if omitted) uses native byte order, native size, and native alignment (with padding, matching your platform's C compiler). = uses native byte order and size but no alignment padding. < forces little-endian, > forces big-endian, and ! forces big-endian ("network" byte order, standard for most network protocols) — the last three always pack with no padding, giving fully predictable, portable output regardless of platform.
For any format you intend to be read on a different machine (a file format, a network protocol), always specify an explicit byte-order character (<, >, or !) rather than relying on the platform-dependent default @ — otherwise your packed bytes are only self-consistent on machines with the same native endianness.
import struct
value = 1
print(struct.pack("<I", value)) # b'\x01\x00\x00\x00' — little-endian
print(struct.pack(">I", value)) # b'\x00\x00\x00\x01' — big-endian
print(struct.pack("!I", value)) # b'\x00\x00\x00\x01' — network byte order (big-endian)pack and unpack
struct.pack(fmt, v1, v2, ...) takes the format string plus one positional value per field and returns the packed bytes. struct.unpack(fmt, buffer) takes the format string and a bytes-like object of exactly the right size, and returns a tuple of the unpacked values — always a tuple, even for a single value.
struct.unpack raises struct.error if the buffer's length doesn't exactly match what the format string expects — it never silently truncates or pads.
import struct
packed = struct.pack("<Hf", 300, 98.6)
print(packed)
(count, temperature) = struct.unpack("<Hf", packed)
print(count, temperature)
# unpack always returns a tuple, even for one field
(single,) = struct.unpack("<I", struct.pack("<I", 42))
print(single) # 42calcsize: Knowing the Byte Size in Advance
struct.calcsize(fmt) returns the exact number of bytes a given format string packs to (or expects to unpack) — useful for reading a fixed-size binary record from a file without hardcoding the byte count separately from the format string.
import struct
fmt = "<IHf" # uint32, uint16, float32
size = struct.calcsize(fmt)
print(size) # 10 bytes (4 + 2 + 4)
with open("records.bin", "rb") as f:
chunk = f.read(size)
record = struct.unpack(fmt, chunk)
print(record)Reading Fixed Binary Layouts from Files
The most common real-world use: parsing a binary file made of fixed-size records or a header followed by a body. Combine calcsize with chunked file reading (a related concept, covered in file iteration & buffering) to loop through records without hardcoding sizes twice.
import struct
RECORD_FMT = "<Iff" # id (uint32), x (float32), y (float32)
RECORD_SIZE = struct.calcsize(RECORD_FMT)
def read_records(path):
records = []
with open(path, "rb") as f:
while chunk := f.read(RECORD_SIZE):
if len(chunk) < RECORD_SIZE:
raise ValueError("truncated record at end of file")
records.append(struct.unpack(RECORD_FMT, chunk))
return recordsWorking with Struct Objects (Compiled Formats)
For repeated packing/unpacking with the same format string (e.g. inside a loop processing many records), struct.Struct(fmt) precompiles the format once into an object with .pack(), .unpack(), and .size — slightly more efficient than calling the module-level functions repeatedly, and it keeps the format string defined in one place.
import struct
record_struct = struct.Struct("<Iff")
print(record_struct.size) # 12
packed = record_struct.pack(1, 3.5, -2.1)
values = record_struct.unpack(packed)
print(values) # (1, 3.5, -2.099999...)Parsing a Real Binary Header: Worked Example
A concrete illustration: reading a simplified BMP image file header, which starts with a fixed-layout structure documented (in the BMP spec) as: 2-byte magic bytes "BM", a 4-byte little-endian file size, 4 reserved bytes, and a 4-byte little-endian offset to the pixel data.
import struct
BMP_HEADER_FMT = "<2sIHHI" # magic, filesize, reserved1, reserved2, data_offset
HEADER_SIZE = struct.calcsize(BMP_HEADER_FMT) # 14 bytes
def read_bmp_header(path):
with open(path, "rb") as f:
raw = f.read(HEADER_SIZE)
magic, filesize, _r1, _r2, data_offset = struct.unpack(BMP_HEADER_FMT, raw)
if magic != b"BM":
raise ValueError("not a BMP file")
return {"filesize": filesize, "data_offset": data_offset}Protocol Parsing: Network Header Example
Network protocols are commonly specified in terms of fixed-width, big-endian ("network order") header fields — struct with "!" is the standard tool for building and parsing them in Python.
import struct
# A toy message header: 2-byte message type, 4-byte payload length
HEADER_FMT = "!HI"
HEADER_SIZE = struct.calcsize(HEADER_FMT)
def build_message(msg_type: int, payload: bytes) -> bytes:
header = struct.pack(HEADER_FMT, msg_type, len(payload))
return header + payload
def parse_header(data: bytes):
msg_type, length = struct.unpack(HEADER_FMT, data[:HEADER_SIZE])
return msg_type, lengthCommon Mistakes with struct
The recurring pitfalls: forgetting an explicit byte-order character and getting platform-dependent output; mismatching the format string's field count against the number of arguments passed to pack; passing a buffer of the wrong length to unpack; and confusing s (a fixed-length byte string field, needing a count prefix like "10s") with c (a single byte, repeated by a count prefix like "10c" to get ten separate one-byte fields).
import struct
# "10s" = one 10-byte string field
print(struct.unpack("10s", b"hello\x00\x00\x00\x00\x00")) # (b'hello\x00\x00\x00\x00\x00',)
# "10c" = ten separate 1-byte fields
print(struct.unpack("10c", b"hello\x00\x00\x00\x00\x00")) # (b'h', b'e', b'l', ..., b'\x00')Key Takeaways
struct converts between Python values and fixed-layout binary bytes, using a compact format-string mini-language mirroring C types.
Always specify an explicit byte-order character (<, >, or !) for any format meant to be portable across machines — the default is platform-dependent.
struct.calcsize(fmt) gives the exact byte size a format expects, letting you read fixed-size chunks from a file without duplicating the size elsewhere.
unpack always returns a tuple (even for one value) and raises struct.error if the buffer length doesn't exactly match the format.
struct.Struct(fmt) precompiles a format for efficient repeated use, such as parsing many fixed-size records in a loop.
's' with a count prefix is one fixed-length byte-string field; 'c' with a count prefix is that many separate single-byte fields — they are not interchangeable.
packed = struct.pack("I", 1) # uses default native/platform byte order packed = struct.pack("<I", 1) # explicit little-endian, portable across machines
Omitting a byte-order character uses the platform's native order and alignment (format character '@' implied); data meant to be read elsewhere needs an explicit '<', '>', or '!' for predictable, portable results.
struct.unpack("iif", some_10_byte_buffer) # struct.error: unpack requires 12 bytes expected = struct.calcsize("iif") if len(some_buffer) != expected: raise ValueError(f"expected {expected} bytes, got {len(some_buffer)}") struct.unpack("iif", some_buffer) unpack requires the buffer length to exactly match the format string's total size; checking with calcsize first (or catching struct.error) avoids a confusing crash on malformed input.
struct.pack("10c", b"hello") # struct.error: expects 10 separate byte arguments, not one 5-byte string struct.pack("10s", b"hello") # pads/truncates a single bytes object to a 10-byte field
'c' repeated by a count means that many individual 1-byte fields (each needing its own argument), while 's' with a count means one fixed-length byte-string field taking a single bytes argument — mixing them up is a common source of struct.error.
What problem does struct solve that json or pickle don't?
struct converts Python values to and from raw, fixed-layout binary bytes matching C-compatible type sizes and byte order — the format used by binary file formats, network protocol headers, and hardware interfaces, none of which json or pickle (which use their own self-describing formats) can produce or parse.
Why is it important to specify an explicit byte-order character like '<' or '>' when packing data meant to be shared across machines?
Without one, struct uses the platform's native byte order and alignment (with padding), which varies between machines and architectures. An explicit '<' (little-endian), '>' (big-endian), or '!' (network/big-endian) produces byte-for-byte identical, portable output regardless of which machine packs or unpacks it.
What does struct.calcsize(fmt) return, and why is it useful when reading binary files?
It returns the exact number of bytes the given format string packs to or expects when unpacking. It's useful for reading fixed-size binary records from a file (f.read(struct.calcsize(fmt))) without hardcoding or duplicating the byte count separately from the format definition.
Why does struct.unpack always return a tuple, even when the format string has only one field?
unpack's return type is consistent regardless of field count for a predictable interface; for a single field you unpack it with a trailing comma, e.g. (value,) = struct.unpack('<I', data), to pull the single element out of the one-tuple.
What is the difference between the format characters 's' and 'c' when each is preceded by a count like 10?
'10s' means one fixed-length, 10-byte string field (takes/returns a single bytes object of length 10). '10c' means ten separate 1-byte fields (takes/returns ten individual one-byte bytes objects) — they parse the same 10 raw bytes very differently.
What exception does struct raise when the buffer passed to unpack doesn't match the format's expected size, and how would you avoid it?
It raises struct.error. You avoid it by reading exactly struct.calcsize(fmt) bytes before calling unpack (e.g. f.read(calcsize(fmt))), or by validating the buffer's length before attempting to unpack it.
Write a function that packs a sequence of (id: int, value: float) pairs into a single bytes object using a fixed little-endian layout of one uint32 and one float32 per pair.
import struct def pack_records(records: list[tuple[int, float]]) -> bytes: fmt = "<If" packer = struct.Struct(fmt) return b"".join(packer.pack(rec_id, value) for rec_id, value in records) struct.Struct('<If') precompiles the fixed little-endian layout (uint32 id + float32 value) once; packing each record and joining the resulting bytes objects produces one contiguous buffer with records laid out back to back at a known, fixed stride.
What does the second print statement output, and why?
(65536,) — data was packed as big-endian (b'\x00\x00\x01\x00'), but unpacked as little-endian, so the byte order is reinterpreted backwards: the byte that meant the '256' place value is now read as the highest-order byte, giving 65536 instead of 256. This illustrates why matching byte order between pack and unpack is essential.
Write a function that reads a sequence of fixed-size binary records from a file (format: uint32 id, uint16 flags, little-endian) and returns them as a list of (id, flags) tuples, raising ValueError on a truncated final record.
import struct def read_records(path: str) -> list[tuple[int, int]]: fmt = "<IH" size = struct.calcsize(fmt) records = [] with open(path, "rb") as f: while chunk := f.read(size): if len(chunk) < size: raise ValueError("truncated record at end of file") records.append(struct.unpack(fmt, chunk)) return records
Reading in exact struct.calcsize(fmt)-sized chunks in a loop matches each read to one full record; checking the chunk length before unpacking catches a partial final record (fewer bytes than expected) and raises a clear error instead of letting struct.unpack fail with a less informative struct.error.
import struct
data = struct.pack(">I", 256)
print(data)
print(struct.unpack("<I", data))