CSV

Reading and writing comma-separated value files correctly with the csv module — quoting, delimiters, dict-based rows, and the newline pitfall almost everyone hits once.

CSV (comma-separated values) looks trivial — just split on commas, right? — until you meet a field that contains a comma, a quote, or an embedded newline. The csv module implements the real dialect rules (quoting and escaping) so you don't hand-roll a parser that breaks on the first messy row of real-world data.

Why Not Just Split on Commas

A naive line.split(",") breaks as soon as a field itself contains a comma (e.g. "Smith, John" as a name) or a newline inside a quoted field. Real CSV files quote such fields (typically with double quotes) and escape embedded quotes by doubling them. The csv module implements this quoting/escaping logic correctly, matching what spreadsheet programs like Excel and Google Sheets actually produce and expect.

A quoted field containing a comma and an escaped quote:
name,note
"Smith, John","Said ""hello"" to everyone"

Naive split(",") would incorrectly break "Smith, John" into two fields.

Basic Syntax: reader and writer

csv.reader(file_obj) wraps an open file and yields each row as a list of strings. csv.writer(file_obj) wraps an open file and provides .writerow() / .writerows() to write rows, handling quoting automatically wherever a field needs it.

All fields read via csv.reader are strings — even the 36 above comes back as '36', not an int. You must convert numeric/typed fields yourself.

import csv

# Writing
with open("people.csv", "w", newline="", encoding="utf-8") as f:
    writer = csv.writer(f)
    writer.writerow(["name", "age", "city"])
    writer.writerows([
        ["Ada Lovelace", 36, "London"],
        ["Grace Hopper", 85, "New York"],
    ])

# Reading
with open("people.csv", newline="", encoding="utf-8") as f:
    reader = csv.reader(f)
    header = next(reader)
    for row in reader:
        print(row)   # each row is a list of strings, e.g. ['Ada Lovelace', '36', 'London']

The newline='' Requirement

This is the single most common CSV mistake in Python: when opening a file for csv reading or writing, always pass newline="" to open(). CSV's own quoting rules handle embedded newlines within quoted fields, and they conflict with Python's universal-newline translation if you leave it enabled — on Windows in particular, this can produce blank extra rows or mangled line endings.

# Correct — csv module manages line endings itself
with open("data.csv", "w", newline="", encoding="utf-8") as f:
    writer = csv.writer(f)
    ...

# Wrong on Windows — can insert extra blank lines between rows
with open("data.csv", "w", encoding="utf-8") as f:
    writer = csv.writer(f)
    ...

DictReader and DictWriter

csv.DictReader reads each row as an OrderedDict-like mapping (a plain dict since 3.8, preserving column order) keyed by the header row's column names — much more readable than indexing rows by position. csv.DictWriter is the inverse: you give it fieldnames up front and write dicts, which it converts to correctly ordered rows.

DictWriter raises ValueError if a row dict has a key not present in fieldnames, unless you pass extrasaction="ignore" to silently drop unexpected keys instead.

import csv

# DictReader — access fields by column name, not position
with open("people.csv", newline="", encoding="utf-8") as f:
    reader = csv.DictReader(f)
    for row in reader:
        print(row["name"], row["age"])   # row is a dict: {'name': ..., 'age': ..., 'city': ...}

# DictWriter — must declare fieldnames; writeheader() writes the column row
with open("out.csv", "w", newline="", encoding="utf-8") as f:
    fieldnames = ["name", "age", "city"]
    writer = csv.DictWriter(f, fieldnames=fieldnames)
    writer.writeheader()
    writer.writerow({"name": "Ada Lovelace", "age": 36, "city": "London"})

Delimiters and Custom Dialects

Not all "CSV-like" files use commas — tab-separated (TSV), semicolon-separated (common in European locales where comma is the decimal separator), and pipe-separated files are common. Pass delimiter=";" (or any single character) to reader/writer/DictReader/DictWriter to handle these.

For a combination of settings you reuse often, register a named dialect once with csv.register_dialect instead of repeating the keyword arguments everywhere.

import csv

with open("euro_data.csv", newline="", encoding="utf-8") as f:
    reader = csv.reader(f, delimiter=";")
    for row in reader:
        print(row)

with open("data.tsv", newline="", encoding="utf-8") as f:
    reader = csv.reader(f, delimiter="\t")
    for row in reader:
        print(row)

import csv

csv.register_dialect("pipes", delimiter="|", quoting=csv.QUOTE_MINIMAL)

with open("data.psv", newline="", encoding="utf-8") as f:
    reader = csv.reader(f, dialect="pipes")
    for row in reader:
        print(row)

Quoting Modes

The quoting parameter controls when the writer wraps fields in quotes. QUOTE_MINIMAL (the default) quotes only fields that need it (containing the delimiter, quote character, or a line terminator). QUOTE_ALL quotes every field unconditionally. QUOTE_NONNUMERIC quotes all non-numeric fields, and on reading, converts unquoted fields to float. QUOTE_NONE disables quoting entirely, requiring an escape character for any delimiter that appears in data.

import csv, io

buf = io.StringIO()
writer = csv.writer(buf, quoting=csv.QUOTE_ALL)
writer.writerow(["Ada", 36, "London"])
print(buf.getvalue())
# "Ada","36","London"

buf2 = io.StringIO()
writer2 = csv.writer(buf2, quoting=csv.QUOTE_MINIMAL)
writer2.writerow(["Ada", 36, "London"])
print(buf2.getvalue())
# Ada,36,London

Handling Embedded Quotes and Special Characters

The default quote character is ". When a field itself contains a quote character, the standard CSV convention (which csv implements by default) is to double it inside the quoted field. You never need to do this manually — the writer handles it automatically.

import csv, io

buf = io.StringIO()
writer = csv.writer(buf)
writer.writerow(['She said "hi"', "a,b"])
print(buf.getvalue())
# "She said ""hi""","a,b"

buf.seek(0)
reader = csv.reader(buf)
row = next(reader)
print(row)   # ['She said "hi"', 'a,b'] — correctly un-escaped on read

Large Files: Streaming Row by Row

Like a plain text file, csv.reader and DictReader are iterators that yield one row at a time — they don't load the whole file into memory. This is the same memory-aware pattern as ordinary line iteration (a related concept), applied to structured rows, so processing a multi-gigabyte CSV works the same way as a small one.

import csv

total_amount = 0.0
with open("transactions.csv", newline="", encoding="utf-8") as f:
    reader = csv.DictReader(f)
    for row in reader:
        total_amount += float(row["amount"])   # process one row at a time
print(total_amount)

Data Validation While Reading

CSV carries no type information and no schema enforcement — every field is a string, and a malformed row (wrong column count, invalid number format) only fails when you try to use it. Validate and convert explicitly as you read, and decide deliberately how to handle bad rows (skip, log, or raise).

import csv

def load_valid_rows(path):
    valid_rows = []
    with open(path, newline="", encoding="utf-8") as f:
        reader = csv.DictReader(f)
        for line_num, row in enumerate(reader, start=2):  # header is line 1
            try:
                age = int(row["age"])
                if age < 0 or age > 150:
                    raise ValueError("age out of plausible range")
                valid_rows.append({**row, "age": age})
            except (ValueError, KeyError) as e:
                print(f"Skipping invalid row at line {line_num}: {e}")
    return valid_rows

Edge Cases: Ragged Rows, Empty Files, BOM

A row with fewer fields than the header leaves DictReader filling the missing keys with None (or the restval you configure); a row with extra fields collects them into a list under the special key None (or restkey if configured). An empty file yields zero rows with no error. Files saved by some Windows tools (notably Excel) begin with a UTF-8 byte-order mark (BOM) — open with encoding="utf-8-sig" instead of "utf-8" to strip it automatically, or the BOM ends up prepended to your first column's header name.

import csv

with open("excel_export.csv", newline="", encoding="utf-8-sig") as f:
    reader = csv.DictReader(f)
    print(reader.fieldnames)   # clean names, no leading '\ufeff' on the first one

Key Takeaways

Never hand-roll CSV parsing with str.split(','); the csv module implements real quoting/escaping rules that a naive split breaks on.

Always open files for csv I/O with newline='' — omitting it can corrupt line endings, especially on Windows.

DictReader/DictWriter access fields by column name instead of position, which is far more readable and resilient to column reordering.

Every field read from CSV is a string; convert types (int, float, date, etc.) explicitly and validate as you go.

delimiter and quoting parameters (or a registered dialect) adapt the module to TSV, semicolon-separated, and other CSV-like formats.

csv.reader/DictReader stream rows one at a time, making large-file processing memory-efficient by default.

with open("data.csv") as f: for line in f: fields = line.strip().split(",") # breaks on quoted fields with commas import csv with open("data.csv", newline="", encoding="utf-8") as f: reader = csv.reader(f) for fields in reader:...

Naive splitting on commas fails as soon as a field is quoted and contains a comma or embedded newline; csv.reader implements the real quoting rules correctly.

with open("data.csv", "w") as f: # missing newline="" writer = csv.writer(f) writer.writerow(["a", "b"]) with open("data.csv", "w", newline="") as f: writer = csv.writer(f) writer.writerow(["a", "b"])

Without newline='', Python's universal-newline translation can conflict with csv's own line-ending handling, producing extra blank rows or mangled line endings, especially on Windows.

total = 0 for row in csv.DictReader(f): total += row["amount"] # TypeError: can't add str + int total = 0 for row in csv.DictReader(f): total += float(row["amount"])

Every field from csv is a string regardless of how it looks; numeric fields must be explicitly converted before arithmetic.

Why is str.split(',') an unreliable way to parse a CSV file?

It breaks on any field that itself contains a comma or embedded newline, which real CSV files handle by quoting such fields and escaping embedded quotes by doubling them. The csv module implements those quoting/escaping rules correctly; a naive split does not.

What does passing newline='' to open() do for CSV files, and why is it necessary?

It disables Python's universal-newline translation at the text-layer, letting the csv module handle line endings (including embedded newlines inside quoted fields) itself according to the CSV format's own rules — omitting it can cause extra blank rows or corrupted line endings, particularly on Windows.

What is the main advantage of DictReader over plain csv.reader?

DictReader lets you access fields by column name (row['name']) rather than by numeric position (row[0]), which is more readable and stays correct even if the column order in the source file changes, as long as the header names match.

What type are all fields when read via csv.reader or DictReader, and what must you do to use them as numbers?

All fields are plain strings, regardless of how they look (e.g. '42' not 42). You must explicitly convert them with int(), float(), or another parser before doing arithmetic or numeric comparisons.

How would you read a semicolon-separated file with the csv module?

Pass delimiter=';' to csv.reader, csv.writer, csv.DictReader, or csv.DictWriter — the module isn't hardcoded to commas and accepts any single-character delimiter.

What happens when DictWriter.writerow() is called with a dict containing a key not in the declared fieldnames?

It raises ValueError by default, since DictWriter doesn't know which output column an unexpected key should go to. Passing extrasaction='ignore' to DictWriter's constructor makes it silently drop unexpected keys instead of raising.

Write a function that reads a CSV file with a header row and returns the sum of a named numeric column, skipping rows where that column can't be converted to float.

import csv def sum_column(path: str, column: str) -> float: total = 0.0 with open(path, newline="", encoding="utf-8") as f: reader = csv.DictReader(f) for row in reader: try: total += float(row[column]) except (ValueError, KeyError): continue return total

DictReader gives named access to the target column; wrapping the float conversion in try/except lets the function skip malformed or missing values gracefully rather than crashing on the first bad row.

What does this print?

'"a,b","c""d"'\r\n — the first field is quoted because it contains the delimiter (comma), and the second field is quoted with its embedded double-quote doubled ("") because it contains the quote character itself; csv.writer also appends \r\n as its default line terminator.

Write a function that converts a list of dicts (with consistent keys) into a CSV file, using DictWriter and writing a header row.

import csv def write_records(path: str, records: list[dict]) -> None: if not records: return fieldnames = list(records[0].keys()) with open(path, "w", newline="", encoding="utf-8") as f: writer = csv.DictWriter(f, fieldnames=fieldnames) writer.writeheader() writer.writerows(records)

DictWriter needs fieldnames up front to know the column order; taking them from the first record's keys, calling writeheader() first, then writerows() for the rest produces a correctly ordered CSV with a header row matching the dict keys.

import csv, io
buf = io.StringIO()
w = csv.writer(buf)
w.writerow(["a,b", 'c"d'])
print(buf.getvalue())