Bytes & Bytearray

When text stops being text — bytes and bytearray are Python's tools for handling raw binary data, network protocols, and file formats byte by byte.

bytes and bytearray are Python's sequence types for raw binary data: fixed sequences of integers in the range 0-255, with no inherent character encoding. bytes is immutable (like str, but for binary data); bytearray is its mutable counterpart, letting you modify contents in place — useful for building up or transforming binary buffers efficiently, such as when parsing network protocols or binary file formats.

What bytes and bytearray Represent

Both types are sequences of small integers, each between 0 and 255 (one byte's worth of values). Unlike str, they carry no notion of 'characters' or encoding — a bytes object is exactly what its name says, a block of bytes, whether it represents encoded text, an image file, a network packet, or anything else binary.

data = b"hello"
type(data)          # <class 'bytes'>
list(data)            # [104, 101, 108, 108, 111] -- each item is an int 0-255
data[0]                # 104 (not 'h' -- indexing a bytes object gives an int)

Creating bytes and bytearray

A bytes literal uses the b prefix, restricted to ASCII characters and escape sequences (\xNN for arbitrary byte values). Both types can also be constructed from an encoded string, an iterable of integers, or a size (which zero-fills).

b1 = b"hello"
b2 = "héllo".encode("utf-8")
b3 = bytes([104, 101, 108, 108, 111])
b4 = bytes(5)                # b'\x00\x00\x00\x00\x00' -- 5 zero bytes

ba = bytearray(b"hello")
ba2 = bytearray(5)             # 5 zero bytes, mutable
ba3 = bytearray(range(65, 70)) # bytearray(b'ABCDE')

Immutable Versus Mutable Binary Data

bytes cannot be changed after creation — like str, any 'modifying' operation returns a new object. bytearray supports item assignment, slice assignment, append(), extend(), and other in-place mutation methods, much like a list of small integers. Choose bytes for data that shouldn't change (safe to share, hashable, usable as a dict key), and bytearray when you're incrementally building or editing a binary buffer and want to avoid repeated allocation.

bytes is hashable (usable as a dict key or set member) precisely because it's immutable; bytearray is not hashable, for the same reason a list isn't.

data = bytearray(b"hello")
data[0] = 72          # ord('H')
print(data)            # bytearray(b'Hello')

data.append(33)        # ord('!')
print(data)             # bytearray(b'Hello!')

immutable = bytes(data)
# immutable[0] = 74   # TypeError: 'bytes' object does not support item assignment

Indexing Behavior: Bytes vs Slices

Indexing a single position with [] returns an int (the byte's numeric value, 0-255) — this is a common surprise coming from str, where indexing returns a one-character string. Slicing, however, returns another bytes (or bytearray) object, not a list of ints, which matches str slicing behavior.

data = b"hello"
data[0]        # 104          (an int)
data[0:2]      # b'he'         (still bytes)
data[-1]       # 111           (an int, last byte)
for byte in data:
    print(byte, end=" ")   # 104 101 108 108 111

Encoding and Decoding Text

The bridge between str and bytes is always explicit: str.encode(encoding) produces bytes, bytes.decode(encoding) produces a str. bytearray supports .decode() too, but not .encode() (it isn't text to begin with).

text = "café"
raw = text.encode("utf-8")     # b'caf\xc3\xa9'
back = raw.decode("utf-8")      # 'café'

buf = bytearray(raw)
buf.decode("utf-8")               # 'café' -- bytearray decodes the same way

String-Like Methods Available on bytes

bytes and bytearray support many of the same methods as str — split(), join(), strip(), replace(), startswith(), find() — because both are sequence types over a similar conceptual shape. The key difference is that arguments and results are bytes-like objects, not str; you cannot mix a bytes method call with a plain str argument.

line = b"name=Ada;role=Engineer"
for pair in line.split(b";"):
    key, _, value = pair.partition(b"=")
    print(key, value)
# b'name' b'Ada'
# b'role' b'Engineer'

b",".join([b"a", b"b", b"c"])   # b'a,b,c'

Building Binary Data with bytearray

bytearray is well-suited to incrementally assembling binary output — for example, constructing a message buffer byte by byte or field by field — since appending or extending it doesn't require creating a new object each time the way repeated bytes concatenation would.

buffer = bytearray()
buffer.extend(b"HDR")
buffer.append(0x01)             # a single byte, version number
buffer.extend((100).to_bytes(2, "big"))   # 2-byte length field
print(bytes(buffer))              # b'HDR\x01\x00d'

Integers and Bytes: to_bytes() / from_bytes()

int.to_bytes(length, byteorder, signed=False) converts an integer into a fixed-length bytes object — essential for binary protocols and file formats that encode numbers as raw byte sequences. int.from_bytes(data, byteorder, signed=False) reverses it. byteorder is 'big' (most significant byte first) or 'little' (least significant byte first) and must match whatever the protocol or file format specifies.

(1000).to_bytes(2, byteorder="big")      # b'\x03\xe8'
(1000).to_bytes(2, byteorder="little")   # b'\xe8\x03'

int.from_bytes(b"\x03\xe8", byteorder="big")   # 1000

Binary Protocols and File I/O (Concept)

Reading a file in binary mode (open(path, "rb")) yields bytes directly, with no decoding — the right approach for images, archives, executables, or any format that is not plain text, and also the right starting point when you plan to parse a structured binary protocol yourself. Network sockets similarly send and receive raw bytes. The standard library's struct module builds on this to pack/unpack fixed binary layouts (multiple typed fields) in one call, which is typically preferable to manual to_bytes()/from_bytes() calls once a format has several fields.

from pathlib import Path

data = Path("image.png").read_bytes()
print(data[:8])   # PNG file signature bytes, e.g. b'\x89PNG\r\n\x1a\n'

memoryview: A Brief Preview

For large binary buffers, slicing bytes/bytearray copies data, which can be wasteful. memoryview provides a zero-copy view over an object that supports the buffer protocol (including bytes and bytearray), letting you slice and inspect without duplicating memory. This is covered in depth as its own topic; the key link here is that memoryview(some_bytearray) lets you edit the underlying bytearray's contents through the view.

Key Takeaways

bytes and bytearray hold raw binary data as sequences of integers 0-255, with no inherent text encoding.

bytes is immutable and hashable; bytearray is mutable (supports item/slice assignment, append, extend) and not hashable.

Indexing a single position returns an int; slicing returns another bytes/bytearray object.

Convert between text and binary explicitly with str.encode() and bytes.decode() — never assume an implicit conversion.

int.to_bytes()/int.from_bytes() convert integers to/from fixed-length byte sequences for binary protocols; byteorder must match the format's specification.

Open files in binary mode ('rb'/'wb') for any non-text data to get/give raw bytes with no decoding.

data[0] == 'h' # comparing an int to a str -- always False data[0] == ord('h') # or: data[0:1] == b'h'

Indexing a bytes object with a single position returns an int (the byte value), not a one-character string, so comparing it directly to a str literal is always False.

immutable_data[0] = 65 # TypeError on a bytes object mutable_data = bytearray(immutable_data) mutable_data[0] = 65 bytes is immutable, just like str -- item assignment is not supported. Convert to bytearray first if in-place modification is needed.

b"hello" + "world" # TypeError: can't concat str to bytes b"hello" + "world".encode("utf-8") bytes and str cannot be mixed directly, just as with any other bytes/str operation -- encode the str to bytes (or decode the bytes to str) before combining them.

value.to_bytes(2, byteorder="little") # producing little-endian when the protocol expects big-endian value.to_bytes(2, byteorder="big") # match whatever the spec/protocol actually requires byteorder has no universal 'correct' default -- it must match the specific binary format or protocol being produced or consumed; picking the wrong one silently produces bytes in the wrong order rather than raising an error.

What is returned by indexing a single position of a bytes object, e.g. b'hello'[0], and how does this differ from indexing a str?

It returns an int (the byte's numeric value, e.g. 104), not a one-character string. Indexing a str at a single position returns a one-character str instead -- this asymmetry is a common source of bugs when porting code between the two types.

What is the key difference between bytes and bytearray?

bytes is immutable (like str) and hashable; bytearray is mutable, supporting item assignment, append(), extend(), and other in-place operations, but is not hashable.

Why can't you concatenate a bytes object and a str directly with +?

Python 3 treats bytes and str as distinct, non-interchangeable types with no implicit conversion between them -- concatenating them raises TypeError. You must explicitly encode the str to bytes or decode the bytes to str first.

What does int.to_bytes(length, byteorder) do, and why does byteorder matter?

It converts an integer into a fixed-length bytes object. byteorder ('big' or 'little') determines whether the most-significant or least-significant byte comes first in the output -- it must match whatever binary format or protocol the bytes are meant to conform to, since the two orderings produce different byte sequences for the same number.

Why is bytearray often preferred over bytes when incrementally building up binary data?

bytearray supports in-place mutation (append, extend, slice assignment) without creating a new object on every change, whereas repeatedly concatenating immutable bytes objects allocates a new object each time -- similar to why list is preferred over repeated tuple concatenation for incremental building.

Why is bytes hashable but bytearray is not?

Hashability requires that an object's value never change after creation, since a hash is typically computed from that value and used to place the object in hash-based structures like sets and dict keys. bytes is immutable so this holds; bytearray is mutable, so allowing it to be hashed could let its hash become stale/inconsistent, exactly the same reason list is unhashable.

How would you read the raw bytes of an image file without any text decoding?

Open the file in binary mode, e.g. Path('image.png').read_bytes() or open('image.png', 'rb').read() -- binary mode returns bytes directly with no encoding/decoding applied, appropriate for any non-text file.

Write a function xor_bytes(data, key) that returns a new bytes object where each byte of data is XORed with the corresponding byte of key, cycling key if it's shorter than data.

def xor_bytes(data: bytes, key: bytes) -> bytes: return bytes(b ^ key[i % len(key)] for i, b in enumerate(data)) encrypted = xor_bytes(b"hello", b"key") decrypted = xor_bytes(encrypted, b"key") print(decrypted) # b'hello' -- XOR is its own inverse

Iterate with enumerate() to get each byte's index and value, XOR it with key[i % len(key)] (cycling through the key), and build a new bytes object from the generator of resulting ints -- bytes() accepts any iterable of ints 0-255.

What does this code print?

It prints bytearray(b'aXc') then b'aXc'. Item assignment on a bytearray mutates it in place (index 1, originally 'b', becomes 'X'), and bytes(buf) creates an immutable copy with the same content, shown without the 'bytearray(...)' wrapper.

Write a function parse_header(data) that takes a 4-byte bytes object where the first 2 bytes are a big-endian unsigned length and the next 2 bytes are a big-endian unsigned version, and returns a (length, version) tuple.

def parse_header(data: bytes) -> tuple[int, int]: length = int.from_bytes(data[0:2], byteorder="big") version = int.from_bytes(data[2:4], byteorder="big") return length, version header = (500).to_bytes(2, "big") + (2).to_bytes(2, "big") print(parse_header(header)) # (500, 2)

Slice out each 2-byte field and convert with int.from_bytes(slice, byteorder='big') -- slicing a bytes object returns another bytes object, which from_bytes() accepts directly.

What does this print?

It prints 'Hello' then b'He'. bytes([72, 101, 108, 108, 111]) builds a bytes object from those integer code values (which spell 'Hello' in ASCII), decode('ascii') converts it to a str, and slicing [:2] returns the first two bytes as a new bytes object, not a str.

buf = bytearray(b"abc")
buf[1] = ord("X")
print(buf)
print(bytes(buf))

data = bytes([72, 101, 108, 108, 111])
print(data.decode("ascii"))
print(data[:2])