AST

The ast standard-library module exposes Python's parsed syntax tree as ordinary Python objects, letting you inspect, analyze, and even rewrite code programmatically.

Every piece of Python source code the interpreter runs is first parsed into an Abstract Syntax Tree (AST): a tree of nodes representing the program's grammatical structure. The standard-library ast module exposes that tree directly as inspectable Python objects, and provides tools to walk it, analyze it, and transform it — the same technique linters, formatters, and code-analysis tools use under the hood.

What Is the ast Module

ast is a standard-library module that lets you parse Python source into its syntax tree, inspect that tree's nodes programmatically, and optionally rewrite it before compiling it back into runnable bytecode. It gives Python-level access to the same parsing stage the interpreter itself uses (see the related Python Execution Internals topic), rather than requiring you to write your own parser.

Parsing Source with ast.parse

ast.parse(source, filename=..., mode=...) parses a string of Python source and returns the root node of its AST — a Module node for the default mode="exec", or an Expression/Interactive node for the other modes. This does not execute anything; it only parses.

ast.parse returns a Module node containing a list of top-level statement nodes in its .body attribute.

import ast

source = """
def greet(name):
    return f"Hello, {name}!"
"""

tree = ast.parse(source)
print(type(tree))          # <class 'ast.Module'>
print(ast.dump(tree, indent=2))

The Syntax Tree: Nodes and Fields

Every construct in Python's grammar has a corresponding AST node class: FunctionDef, Assign, BinOp, Name, Constant, If, Call, and dozens more, all defined in the ast module. Each node has typed fields describing its parts (for example, BinOp has left, op, and right), and every node carries lineno/col_offset attributes recording its source location — useful for error messages and code-analysis tooling that needs to point back at the original source.

import ast

tree = ast.parse("x = 1 + 2")
assign_node = tree.body[0]
print(type(assign_node))          # <class 'ast.Assign'>
print(assign_node.targets)        # [<ast.Name object ...>]
print(type(assign_node.value))    # <class 'ast.BinOp'>
print(assign_node.value.left.value, assign_node.value.right.value)  # 1 2
print(assign_node.lineno)         # 1

Traversal: ast.walk and NodeVisitor

ast.walk(node) yields every node in the tree in no particular guaranteed order — good for quick searches ('does this tree contain any Call node?'). For structured traversal where you want to react to specific node types as you visit them (and control whether to recurse into children), subclass ast.NodeVisitor and define visit_<NodeType> methods; calling .generic_visit(node) inside one continues the walk into that node's children.

visit_Call fires for every Call node found anywhere in the tree; generic_visit continues traversal into that node's children.

import ast

source = """
def f(x):
    return g(x) + h(x)
"""

class CallCounter(ast.NodeVisitor):
    def __init__(self):
        self.calls: list[str] = []

    def visit_Call(self, node: ast.Call) -> None:
        if isinstance(node.func, ast.Name):
            self.calls.append(node.func.id)
        self.generic_visit(node)   # keep walking into arguments, etc.

tree = ast.parse(source)
counter = CallCounter()
counter.visit(tree)
print(counter.calls)   # ['g', 'h']

Common Node Types

Node               | Represents
--------------------|---------------------------------------
Module              | the whole parsed source file/string
FunctionDef/AsyncFunctionDef | a def / async def statement
ClassDef            | a class statement
Assign / AugAssign  | x = value / x += value
Name                | a bare identifier reference (x)
Constant            | a literal (number, string, True, None, ...)
BinOp / Compare     | a + b, a < b, etc.
Call                | f(args)
If / For / While    | control-flow statements
Return / Yield      | return value / yield value
Import / ImportFrom | import statements

Transforming Trees with NodeTransformer

ast.NodeTransformer is a NodeVisitor subclass whose visit_* methods can return a replacement node (or None to delete the node, or a list of nodes) instead of just observing. This is how source-to-source rewriting tools work: parse, transform the tree, then ast.fix_missing_locations() to patch in valid line/column info for any newly created nodes, and compile the result.

ast.copy_location and ast.fix_missing_locations keep source position metadata valid on synthesized nodes.

import ast

class ConstantFolder(ast.NodeTransformer):
    """Replaces int + int at parse time with a pre-computed constant."""
    def visit_BinOp(self, node: ast.BinOp) -> ast.AST:
        self.generic_visit(node)   # fold children first
        if (
            isinstance(node.op, ast.Add)
            and isinstance(node.left, ast.Constant)
            and isinstance(node.right, ast.Constant)
            and isinstance(node.left.value, int)
            and isinstance(node.right.value, int)
        ):
            return ast.copy_location(ast.Constant(value=node.left.value + node.right.value), node)
        return node

tree = ast.parse("x = 1 + 2")
tree = ConstantFolder().visit(tree)
ast.fix_missing_locations(tree)
print(ast.dump(tree.body[0].value))   # Constant(value=3)

Compiling and Running a Modified Tree

A tree returned by ast.parse (transformed or not) can be handed to compile(tree, filename, mode) exactly as a source string can, producing a runnable code object. This closes the loop: parse to AST, optionally transform, compile back to bytecode, exec() the result.

import ast

tree = ast.parse("print(1 + 2)")
code_obj = compile(tree, "<ast>", "exec")
exec(code_obj)   # prints 3

ast.dump and ast.unparse

ast.dump(node) renders a tree as a readable string for debugging — pass indent=2 for a multi-line, human-friendly layout. ast.unparse(node) (3.9+) does the reverse of parsing: it converts an AST back into equivalent Python source text, useful after a transformation to see (or save) the resulting code.

import ast

tree = ast.parse("x = 1 + 2")
print(ast.unparse(tree))   # x = 1 + 2

Practical Usage: Code Analysis

Real-world uses of ast include: linters checking for style or safety issues (unused imports, bare except:, dangerous eval calls) without executing the code; static analyzers computing metrics (function complexity, import graphs); code-modernization tools that rewrite deprecated syntax; and safely inspecting untrusted code's structure before deciding whether to run it. Third-party tools like Black (a formatter) and Ruff (a linter) are built on this kind of tree analysis, though they use their own parsers for speed rather than the stdlib ast module directly.

import ast

def find_bare_excepts(source: str) -> list[int]:
    """Returns line numbers of bare 'except:' clauses — a common anti-pattern."""
    tree = ast.parse(source)
    return [
        node.lineno
        for node in ast.walk(tree)
        if isinstance(node, ast.ExceptHandler) and node.type is None
    ]

code = """
try:
    risky()
except:
    pass
"""
print(find_bare_excepts(code))   # [4]

Edge Cases and Caveats

ast.literal_eval(node_or_string) safely evaluates a string or AST node containing only Python literals (numbers, strings, tuples, lists, dicts, booleans, None) — it's the recommended safe alternative to eval() for parsing trusted-format but not-trusted-content data (like a config value), because it refuses anything beyond literals rather than executing arbitrary code.

The exact set of AST node classes and their fields can change slightly between Python versions as the grammar evolves (e.g. pattern matching added Match/MatchCase/etc. in 3.10) — code doing deep AST manipulation across versions should be tested against each target version.

Parsing does not perform semantic checks — a tree can be syntactically valid but reference undefined names or misuse types; those errors only surface at compile/run time, not from ast.parse itself.

Best Practices

Use ast.literal_eval instead of eval() whenever you only need to parse literal data, never arbitrary code.

Prefer NodeVisitor/NodeTransformer over manually recursing through node._fields — they handle the traversal boilerplate and are the idiomatic pattern tools in this space use.

Always call ast.fix_missing_locations() after constructing or splicing in new nodes before compiling a transformed tree.

Treat ast as a read/analyze/rewrite tool for source code, not a substitute for actually running it — semantic questions (what will this code do at runtime) generally require execution or a proper static-analysis framework, not just tree inspection.

Complete Example

A minimal but real illustration of the technique linters use: walk the tree, track imports, track name usage, diff the two sets.

import ast

class UnusedImportFinder(ast.NodeVisitor):
    """A simplified illustration: finds top-level 'import x' names never referenced."""
    def __init__(self):
        self.imported: dict[str, int] = {}
        self.used: set[str] = set()

    def visit_Import(self, node: ast.Import) -> None:
        for alias in node.names:
            name = alias.asname or alias.name
            self.imported[name] = node.lineno
        self.generic_visit(node)

    def visit_Name(self, node: ast.Name) -> None:
        self.used.add(node.id)
        self.generic_visit(node)

    def unused(self) -> dict[str, int]:
        return {name: line for name, line in self.imported.items() if name not in self.used}

source = """
import os
import sys

print(sys.argv)
"""

finder = UnusedImportFinder()
finder.visit(ast.parse(source))
print(finder.unused())   # {'os': 2}

Mental Model

Treat the AST as source code's grammar rendered as data you can program against — a FunctionDef node isn't a description of a function, it *is* the parsed function definition, with a name, arguments, and a body that is itself a list of statement nodes, recursively. Walking the tree is the same activity as reading the source top to bottom, just structured as objects instead of text.

Quick Reference

ast.parse(source)              # source -> AST (Module node)
ast.dump(node, indent=2)       # AST -> readable debug string
ast.unparse(node)               # AST -> source text (3.9+)
ast.walk(node)                  # iterate every node, unordered
ast.NodeVisitor                 # subclass; define visit_<Type>
ast.NodeTransformer             # subclass; visit_<Type> returns replacement
ast.fix_missing_locations(node) # patch lineno/col_offset on new nodes
ast.literal_eval(node_or_str)   # safe eval of literals only
compile(tree, filename, mode)   # AST -> code object

Key Takeaways

ast.parse gives you Python's own parsed syntax tree as inspectable objects, without executing anything.

Every grammar construct is a typed node class with fields and source-location attributes.

NodeVisitor observes a tree; NodeTransformer rewrites it, returning replacement nodes from visit_* methods.

A tree (transformed or not) can be compile()d and exec()d just like source text.

Prefer ast.literal_eval over eval() when you only need to parse literal data safely.

This is the technique behind linters, formatters, and code-analysis tools — inspect structure without running the code.

config_value = eval(user_supplied_string) # arbitrary code execution risk import ast config_value = ast.literal_eval(user_supplied_string) # only literals allowed eval() executes arbitrary expressions, including malicious code; ast.literal_eval only accepts literal data structures and raises on anything else.

class Rewriter(ast.NodeTransformer): def visit_Constant(self, node): return ast.Constant(value=node.value * 2) # missing ast.fix_missing_locations before compiling tree = Rewriter().visit(tree) ast.fix_missing_locations(tree) compile(tree, "<ast>", "exec")

New or modified nodes may lack valid lineno/col_offset attributes; compiling without fix_missing_locations can raise or produce misleading tracebacks.

What does ast.parse(source) return, and does it execute the source?

It returns the root AST node (a Module node for the default exec mode) representing the parsed structure of the source; it only parses, it never executes anything.

What is the difference between ast.NodeVisitor and ast.NodeTransformer?

NodeVisitor observes nodes via visit_<Type> methods without changing the tree; NodeTransformer's visit_<Type> methods can return a replacement node, None to delete it, or a list of nodes, actually rewriting the tree.

Why must you call generic_visit(node) inside a visit_* override if you want traversal to continue into that node's children?

Overriding visit_<Type> replaces the default traversal for that node; calling generic_visit explicitly resumes visiting its child nodes, otherwise traversal stops there.

Why is ast.literal_eval safer than eval() for parsing untrusted string data?

literal_eval only accepts literal Python values (numbers, strings, tuples, lists, dicts, booleans, None) and raises on anything else, so it cannot execute arbitrary code, unlike eval which evaluates any valid expression.

What must you do after constructing or splicing new nodes into a tree before compiling it?

Call ast.fix_missing_locations(tree) to patch in valid lineno/col_offset attributes on the new nodes, since compile() requires them.

What does ast.unparse do, and since which version is it available?

It converts an AST back into equivalent Python source text; it's available since Python 3.9.

Give a real-world example of a tool built on the technique the ast module provides.

A linter that flags bare except: clauses or unused imports by walking the parsed tree and checking node types/fields, without ever executing the analyzed code.

Write a function count_function_defs(source) that returns how many function definitions (def, not async def) appear anywhere in the given source, using ast.walk.

import ast def count_function_defs(source: str) -> int: tree = ast.parse(source) return sum(1 for node in ast.walk(tree) if isinstance(node, ast.FunctionDef)) print(count_function_defs("def a(): pass\ndef b(): pass\nclass C:\n def c(self): pass"))

Returns 3 — ast.walk visits every node in the tree regardless of nesting, so it finds a, b, and the method c defined inside class C.

What does this print, given operator precedence?

Prints 'BinOp' then 'Constant BinOp' — the outer node is the multiplication (3 * (...)), whose left side is the Constant 3 and whose right side is itself a BinOp representing (4 + 5), because parentheses group the addition inside the multiplication.

Write a NodeTransformer subclass NameRenamer that renames every occurrence of the identifier 'old' to 'new' in a parsed tree, then unparse the result.

import ast class NameRenamer(ast.NodeTransformer): def visit_Name(self, node: ast.Name) -> ast.Name: if node.id == "old": node.id = "new" return node tree = ast.parse("old = old + 1\nprint(old)") tree = NameRenamer().visit(tree) ast.fix_missing_locations(tree) print(ast.unparse(tree))

Prints 'new = new + 1\nprint(new)' — every ast.Name node with id 'old' is mutated in place to 'new', and unparse renders the modified tree back to source text.

import ast

tree = ast.parse("y = 3 * (4 + 5)")
node = tree.body[0].value
print(type(node).__name__)
print(type(node.left).__name__, type(node.right).__name__)