Optimization Strategies
Once profiling has found the real bottleneck, the actual fix is usually a better algorithm, less redundant work, batching, caching, a vectorized library, or the right concurrency model — in roughly that order of leverage.
Optimization strategies are the concrete moves you make once profiling (see the related Performance Tools topic) has identified an actual bottleneck — never before. This topic surveys the standard toolbox, roughly ordered by how much leverage each typically provides: choosing a better algorithm, eliminating unnecessary work, batching, caching, vectorized libraries, native extensions, and choosing the right concurrency model for the workload.
Optimize Only What Profiling Confirms
Every strategy in this topic assumes you've already identified the actual bottleneck via measurement (cProfile, timeit, tracemalloc — see Performance Tools), not intuition. Applying any of these techniques to code that isn't actually slow adds complexity and maintenance burden for no measurable benefit — this is premature optimization, and it's a real cost, not a free safety margin.
Choose a Better Algorithm
Algorithmic complexity (see the related Performance Fundamentals topic) is almost always the highest-leverage lever available: an O(n log n) or O(n) algorithm beats even a heavily hand-tuned O(n²) one at realistic scale, no matter how much micro-optimization is applied to the O(n²) version. Before reaching for caching, native code, or parallelism, ask whether a fundamentally better algorithm or data structure eliminates the problem outright.
The second version replaces the nested-loop search with a single pass and a set lookup, changing the complexity class from O(n^2) to O(n) — a difference that dwarfs any low-level tuning of the first version.
# O(n^2): checks every pair for a match
def has_pair_summing_to(numbers: list[int], target: int) -> bool:
for i, a in enumerate(numbers):
for b in numbers[i + 1:]:
if a + b == target:
return True
return False
# O(n): a single pass with a set of "complements seen so far"
def has_pair_summing_to_fast(numbers: list[int], target: int) -> bool:
seen: set[int] = set()
for n in numbers:
if target - n in seen:
return True
seen.add(n)
return FalseReduce Unnecessary Work
The fastest way to do something is often to not do it at all. Common sources of unnecessary work: recomputing a value that hasn't changed since it was last computed, iterating a collection multiple times where one pass would do, doing expensive work inside a loop that's invariant across iterations (and could be hoisted outside it), and computing more than the caller actually needs (e.g. sorting an entire list when only the minimum was needed).
The first version normalizes (.strip().lower()) each item twice per matching iteration and re-indexes items[i] repeatedly; the second computes normalized exactly once per item and reuses it.
# recomputes len(items) and does redundant lookups every iteration
def process(items: list[str]) -> list[str]:
results = []
for i in range(len(items)):
if items[i].strip().lower() in ALLOWED_SET:
results.append(items[i].strip().lower())
return results
# hoists the invariant work, computes each derived value once
def process_fast(items: list[str]) -> list[str]:
results = []
for item in items:
normalized = item.strip().lower()
if normalized in ALLOWED_SET:
results.append(normalized)
return resultsBatching
Batching groups many small operations into fewer, larger ones, amortizing fixed per-operation overhead across many items at once. This matters most when each individual operation has meaningful fixed cost independent of its size — a network round-trip, a database query, a subprocess launch — where doing the work one item at a time pays that fixed cost n times instead of once.
executemany sends all the rows in a single round-trip to the database (implementation-dependent, but generally far fewer network/protocol round-trips than one execute() per row), amortizing the fixed per-call cost across the whole batch.
# one database round-trip per row -- fixed per-call overhead paid n times
for record in records:
db.execute("INSERT INTO events (payload) VALUES (?)", (record,))
# one round-trip for the whole batch -- fixed overhead paid once
db.executemany("INSERT INTO events (payload) VALUES (?)", [(r,) for r in records])Caching
Caching (introduced in the Performance Fundamentals topic) avoids recomputing or re-fetching a result already obtained, trading memory for time. functools.lru_cache/cache is the standard-library go-to for memoizing a pure function; for results that must persist across process restarts, or be shared between multiple processes, an external cache (a file, or a third-party in-memory store like Redis) is the appropriate extension, though setting one up is outside this topic's scope.
Caching is only correct when the cached function's result doesn't change between calls with the same arguments in ways your program cares about (e.g. an exchange rate that updates hourly needs a bounded cache lifetime or explicit invalidation, not an unbounded, permanent cache).
from functools import lru_cache
@lru_cache(maxsize=256)
def fetch_exchange_rate(currency_code: str) -> float:
# imagine this makes a network call -- expensive to repeat
return _slow_network_lookup(currency_code)
fetch_exchange_rate("EUR") # first call: does the real (slow) lookup
fetch_exchange_rate("EUR") # second call: returns the cached result instantlyVectorized Libraries
Vectorization means expressing a computation as a single operation over an entire array rather than a Python-level loop over individual elements, letting the heavy lifting happen inside compiled code rather than the interpreter. numpy (third-party, not in the standard library) is the canonical example: numpy_array * 2 computes the entire elementwise multiplication in optimized C, calling into Python exactly once for the whole operation, rather than once per element as a pure-Python loop would.
The numpy version pays Python-interpreter overhead once for the whole array operation, versus once per element for the pure-Python loop — this is why vectorized numeric code is often dramatically faster for large arrays, without needing to hand-optimize the loop itself.
import numpy as np # third-party, not part of the standard library
# pure-Python loop: one Python-level operation (and function-call-ish overhead) per element
values = list(range(1_000_000))
squared = [v * v for v in values]
# numpy: one call into compiled code for the entire array
arr = np.arange(1_000_000)
squared_arr = arr ** 2Native Extensions Overview
For CPU-bound Python code that remains too slow even after algorithmic and vectorization improvements, native extensions move the hot path into compiled code that Python calls into. Cython (third-party) compiles a Python-like source (often with optional type annotations) to C; cffi and the standard-library ctypes module let Python call into existing compiled C libraries directly. This is a significant escalation in complexity (build tooling, compiled artifacts, a less pure-Python codebase) and is generally reached for only after simpler strategies (better algorithm, vectorization) have been tried and profiling still shows the hot path dominating.
Multiprocessing/Async Choices
Once a workload's shape is understood (CPU-bound vs I/O-bound, per the Performance Fundamentals topic), the concurrency model follows: CPU-bound work that's already algorithmically optimal and can be split across independent chunks benefits from multiprocessing (or concurrent.futures.ProcessPoolExecutor) to use multiple cores, since CPython's traditional GIL prevents CPU-bound threads from running Python bytecode in true parallel. I/O-bound work (many concurrent network requests, for instance) benefits from asyncio or threads, which overlap waiting time rather than needing multiple cores — the GIL is not the bottleneck there, since I/O waits release it.
Practical Usage
A practical optimization sequence, applied only after profiling confirms a real bottleneck: (1) is there a better algorithm or data structure? (2) is the code doing redundant or invariant work it doesn't need to repeat? (3) can the operation be batched to amortize fixed overhead? (4) would caching avoid recomputation for repeated inputs? (5) for numeric-heavy work, does a vectorized library apply? (6) as a last resort for a genuinely hot pure-Python path, would a native extension help? (7) does the concurrency model (multiprocessing vs asyncio/threads) match whether the workload is CPU- or I/O-bound?
Edge Cases
Batching too aggressively (e.g. one enormous database transaction instead of many small ones) can trade one problem for another — memory pressure, longer lock hold times, or a single point of failure that discards an entire batch on one bad record. Caching a function whose result depends on more than its arguments (e.g. reads mutable global state, or wall-clock time) silently produces stale or wrong results — only cache functions that are effectively pure with respect to their arguments.
Complete Example
from functools import lru_cache
# 1. algorithm: O(n) membership check via set instead of O(n) list scan per call, n times
def unique_preserving_order(items: list[str]) -> list[str]:
seen: set[str] = set()
result = []
for item in items:
if item not in seen: # O(1) average
seen.add(item)
result.append(item)
return result
# 2. caching: avoid recomputation for a pure, repeatedly-called function
@lru_cache(maxsize=512)
def normalize_key(raw: str) -> str:
return raw.strip().lower()
# 3. batching: one bulk write instead of many individual writes
def save_all(db, records: list[dict]) -> None:
db.executemany(
"INSERT INTO items (key, value) VALUES (:key, :value)",
records,
)
def process_and_save(db, raw_items: list[str]) -> None:
unique_items = unique_preserving_order(raw_items)
records = [{"key": normalize_key(item), "value": item} for item in unique_items]
save_all(db, records) # single batched writeMental Model
Think of optimization strategies as a ladder ranked by leverage: a better algorithm is changing the shape of the mountain you have to climb; batching and caching are taking fewer, bigger trips instead of many small ones; vectorization and native extensions are switching to faster transportation for the trip itself; and choosing multiprocessing vs asyncio is picking the right vehicle for the terrain (a highway for parallel roads, a queue-skipping lane for waiting in line). Climb the ladder from the top — a better algorithm — before reaching for the lower rungs.
Quick Reference
1. Better algorithm/data structure - highest leverage, changes complexity class
2. Reduce unnecessary work - hoist invariants, avoid recomputation
3. Batching - amortize fixed per-operation overhead
4. Caching (lru_cache/cache) - trade memory for avoided recomputation
5. Vectorized libraries (numpy) - push loops into compiled code
6. Native extensions (Cython, ctypes) - last resort for a hot pure-Python path
7. Match concurrency to workload - multiprocessing (CPU-bound) vs asyncio/threads (I/O-bound)
Always: measure first, optimize the confirmed bottleneck onlyKey Takeaways
Apply optimization strategies only to a bottleneck profiling has actually confirmed — otherwise it's premature optimization, adding complexity for no measured benefit.
A better algorithm or data structure is almost always the highest-leverage fix — check this before caching, native code, or parallelism.
Batching amortizes fixed per-operation overhead (network round-trips, subprocess launches) across many items at once.
functools.lru_cache/cache trades memory for avoided recomputation, but only for functions that are effectively pure with respect to their arguments.
Vectorized libraries (numpy, third-party) and native extensions (Cython, ctypes) push hot inner loops into compiled code, at the cost of an added dependency or build complexity.
Match the concurrency model to the workload: multiprocessing for CPU-bound parallelism, asyncio/threads for I/O-bound concurrency.
def fast_search(haystack: set, needle) -> bool: return needle in haystack # O(1) average -- fix the algorithm first # caching only helps if the SAME (haystack, needle) pair repeats often; # converting to a set fixes every call, repeated or not
Caching only pays off for repeated identical calls; it does nothing for the underlying per-call cost when arguments differ each time. Here the real problem is an O(n) list scan — converting to a set fixes every call's complexity, which caching alone cannot do.
for record in one_million_records: db.execute("INSERT INTO t VALUES (?)", (record,)) # 1,000,000 round-trips
BATCH_SIZE = 1000 for i in range(0, len(one_million_records), BATCH_SIZE): chunk = one_million_records[i:i + BATCH_SIZE] db.executemany("INSERT INTO t VALUES (?)", [(r,) for r in chunk])
One round-trip per record pays the fixed per-call overhead a million times; batching in reasonably-sized chunks amortizes that overhead while avoiding the opposite extreme (one enormous transaction) that risks memory pressure or long lock hold times.
@lru_cache(maxsize=None) def current_stock_price(ticker: str) -> float: return fetch_from_api(ticker) # changes constantly -- cached value goes stale forever from functools import lru_cache import time @lru_cache(maxsize=256) def _cached_price(ticker: str, cache_bucket: int) -> float: return fetch_from_api(ticker) def current_stock_price(ticker: str) -> float: # cache_bucket changes every 60 seconds, effectively expiring old entries cache_bucket = int(time.time() // 60) return _cached_price(ticker, cache_bucket)
An unbounded, permanent cache on a function whose real-world result changes over time (a live stock price) returns increasingly stale data forever. Incorporating a time-bucketed argument gives the cache a natural, bounded lifetime instead.
Why should a better algorithm generally be considered before caching or native extensions when addressing a performance bottleneck?
A better algorithm changes the fundamental complexity class (e.g. O(n^2) to O(n)), which at realistic scale outweighs the constant-factor improvements that caching, vectorization, or native code typically provide. Fixing the algorithm addresses every call's cost; caching only helps for repeated identical calls, and native code speeds up the same (possibly still poor) algorithm.
What kind of operation benefits most from batching, and why?
Operations with meaningful fixed cost per call, independent of how much data that call carries — a network round-trip, a database query, a subprocess launch. Batching amortizes that fixed cost across many items in one call instead of paying it once per item, which matters most when the fixed overhead is large relative to the per-item work.
Why is functools.lru_cache an ineffective fix for a function whose underlying algorithm is inefficient?
Caching only avoids repeating identical work for repeated identical arguments; it does nothing to reduce the cost of a call with arguments that haven't been seen before. If the underlying implementation is O(n) or worse per call, every distinct call still pays that full cost regardless of caching.
Why is caching a function like 'current stock price' dangerous with an unbounded, permanent cache?
The real-world value changes over time, but an unbounded cache never re-fetches it once cached, so callers receive an increasingly stale value indefinitely. The cache needs either a bounded lifetime (time-based expiration) or explicit invalidation to remain correct.
Why does vectorized numpy code often outperform an equivalent pure-Python loop for large arrays, even though both ultimately do the 'same' arithmetic?
A pure-Python loop pays interpreter-level overhead (bytecode dispatch, object boxing/unboxing) once per element; a vectorized numpy operation dispatches into compiled C code once for the entire array, so the per-element overhead the interpreter would otherwise pay is largely eliminated.
When is multiprocessing the right choice over asyncio for a performance problem, and vice versa?
Multiprocessing is right for CPU-bound work that can be split into independent chunks, since it uses multiple OS processes (and thus multiple cores) to sidestep CPython's traditional GIL. Asyncio is right for I/O-bound work — many concurrent operations that spend most of their time waiting on external resources — since it overlaps that waiting time within a single process/thread without needing multiple cores.
Why is applying every optimization strategy in this topic to code that hasn't been profiled considered a mistake rather than just extra caution?
Every strategy adds real cost — code complexity, reduced readability, added dependencies, or memory use — in exchange for a speed benefit. Applying them to code that isn't actually a bottleneck pays that cost for no measurable gain, which is the textbook definition of premature optimization, and it can make the codebase harder to maintain without any offsetting benefit.
Profiling shows this function is a hot path, called many times with the same pattern argument but a different items list each time. What's the highest-leverage optimization here, and why?
pattern.strip().lower() is invariant across every iteration of the loop (and across every call with the same pattern), yet it's recomputed on every single item — pure wasted work. Hoisting it out of the loop (normalized_pattern = pattern.strip().lower() computed once, before the loop) removes redundant work with no change in behavior: def count_matches(items, pattern):\n normalized_pattern = pattern.strip().lower()\n return sum(1 for item in items if normalized_pattern in item.strip().lower()). This is 'reduce unnecessary work' — higher leverage here than caching, since items differs on every call so a whole-function cache wouldn't hit anyway.
Write a function batch_insert(db, table: str, rows: list[dict], batch_size: int = 500) -> None that inserts rows into a database in batches of batch_size using executemany, rather than one execute() call per row.
def batch_insert(db, table: str, rows: list[dict], batch_size: int = 500) -> None: if not rows: return columns = list(rows[0].keys()) placeholders = ", ".join(f":{col}" for col in columns) sql = f"INSERT INTO {table} ({', '.join(columns)}) VALUES ({placeholders})" for i in range(0, len(rows), batch_size): chunk = rows[i:i + batch_size] db.executemany(sql, chunk)
Rows are sliced into chunks of batch_size and sent via executemany per chunk, amortizing each database round-trip's fixed overhead across up to batch_size rows instead of paying that overhead once per individual row. Batching in fixed-size chunks (rather than one giant executemany for everything) also bounds memory use and transaction size for very large rows lists.
Is lru_cache(maxsize=None) appropriate here? What property of get_config makes caching either safe or unsafe?
It's appropriate here specifically because the underlying config is stated to change only on restart, not during the process's lifetime — so get_config(key) is effectively pure (same key always yields the same result) for the life of the running process, which is exactly the property caching requires for correctness. If the config could instead change while the process kept running (e.g. hot-reloaded), an unbounded permanent cache would silently serve stale values, and the cache would need either invalidation or a bounded lifetime, similar to the stock-price example in this topic's Common Mistakes.
def count_matches(items, pattern):
count = 0
for item in items:
if pattern.strip().lower() in item.strip().lower():
count += 1
return countimport time
from functools import lru_cache
@lru_cache(maxsize=None)
def get_config(key: str) -> str:
return _load_config_from_disk(key) # config file changes when the app restarts, not while running