Module 01 - Python Performance Engineering

The Benchmark That Changes Everything

Before reading a single word of theory, run this:

import time
import numpy as np

def sum_squares_pure(n: int) -> float:
    """Naive Python - what most engineers write first."""
    total = 0.0
    for i in range(n):
        total += i * i
    return total

# Time it
N = 10_000_000
start = time.perf_counter()
result = sum_squares_pure(N)
elapsed = time.perf_counter() - start
print(f"Pure Python:  {elapsed:.3f}s  →  {result:.0f}")

On a modern laptop this prints something like:

Pure Python:  4.21s  →  333333283333335000000.0

Four seconds for ten million multiplications and additions. Now look at what happens when you change only the implementation strategy - the algorithm stays identical:

from numba import njit
import numpy as np

# Strategy 1: Numba JIT compilation
@njit
def sum_squares_numba(n: int) -> float:
    total = 0.0
    for i in range(n):
        total += i * i
    return total

# Warm up the JIT (first call compiles, subsequent calls run native)
sum_squares_numba(1)

start = time.perf_counter()
result = sum_squares_numba(N)
elapsed = time.perf_counter() - start
print(f"Numba JIT:    {elapsed:.3f}s  →  {result:.0f}")

# Strategy 2: NumPy vectorisation
def sum_squares_numpy(n: int) -> float:
    arr = np.arange(n, dtype=np.float64)
    return np.dot(arr, arr)

start = time.perf_counter()
result = sum_squares_numpy(N)
elapsed = time.perf_counter() - start
print(f"NumPy:        {elapsed:.3f}s  →  {result:.0f}")

Typical output:

Numba JIT:    0.031s  →  333333283333335040.0
NumPy:        0.089s  →  333333283333335040.0

And if you compile the same loop with Cython using typed memoryviews (covered in Lesson 02), you get approximately 0.018s - 233x faster than the naive version.

Strategy	Time	Speedup	Key Requirement
Pure Python	4.21 s	1x	None
NumPy vectorised	0.089 s	47x	Data fits in array form
Numba JIT	0.031 s	136x	Numerical types, no arbitrary objs
Cython typed	0.018 s	233x	C compilation toolchain

The critical lesson: the fastest approach depends entirely on the workload. NumPy is not always fastest. Numba is not always fastest. Cython is not always fastest. This module teaches you how to determine which tool to reach for - and when to reach for none of them because the bottleneck is elsewhere entirely.

What You Will Learn

By the end of this module you will be able to:

• Profile any Python application to find its real bottleneck (not the assumed one)
• Use cProfile, line_profiler, py-spy, and memory_profiler to measure before touching any code
• Accelerate numerical kernels with Cython typed memoryviews and GIL release
• Apply Numba's @njit, @vectorize, and @cuda.jit to loop-heavy scientific code
• Reduce Python memory usage by 3–10x using __slots__, generators, and mmap
• Tune asyncio event loops, connection pools, and backpressure for high-throughput services
• Choose between multiprocessing, Ray, Dask, and Celery for distributed workloads

Prerequisites

Requirement	Level Needed
Python syntax and typing	Comfortable
NumPy basics	Familiar
asyncio fundamentals	Familiar
OS/process/thread concepts	Basic understanding
C compilation (gcc/clang)	Not required

You do not need a GPU. CUDA examples in Lesson 03 are labelled optional.

Why Python Is "Slow" - The Real Explanation

Python's reputation for being slow is accurate but routinely misdiagnosed. Understanding why it is slow tells you exactly when it matters and which tool fixes each class of problem.

1. The Global Interpreter Lock (GIL)

CPython, the reference implementation, has a single lock - the GIL - that must be held to execute Python bytecode. Only one thread executes Python at a time, regardless of how many CPU cores are available.

Thread A running: ───────▓▓▓▓▓▓▓───────▓▓▓▓▓▓▓───────
Thread B waiting: ───────░░░░░░░▓▓▓▓▓▓▓░░░░░░░▓▓▓▓▓▓▓
                         GIL switch interval (~5ms default)

This means Python threads provide no CPU parallelism for Python code. They do release the GIL during I/O operations and during calls into C extensions that are written to release it (NumPy does, during many operations).

Implication: threading.Thread for CPU-bound work is useless. multiprocessing, Ray, and Dask are the real parallelism tools.

2. Dynamic Dispatch

Every attribute access and function call in Python involves a dictionary lookup. When you write x + y:

x + y
  → x.__add__(y)
  → type(x).__dict__['__add__'](x, y)    # dict lookup
  → ... bounds check, refcount update ...
  → actual addition                       # finally

Compare to C: x + y compiles to a single addsd instruction on x86.

Python's dynamism (duck typing, monkey patching, __getattr__ overrides) is what makes it expressive - but every dispatch costs 50–200 ns. Ten million iterations at 50ns each = 500ms of dispatch overhead alone.

3. Object Overhead

Every Python object carries a header:

// Every Python object starts with this (simplified)
typedef struct {
    Py_ssize_t ob_refcnt;   // 8 bytes - reference count
    PyTypeObject *ob_type;  // 8 bytes - pointer to type
} PyObject;

A Python integer takes 28 bytes of memory. A C int64 takes 8 bytes. A list of 1 million Python integers consumes ~28 MB. A NumPy array of 1 million int64 values consumes 8 MB and supports SIMD vectorisation.

4. Interpreter Overhead

CPython executes bytecode via an eval loop - essentially a giant switch statement over opcodes. Each bytecode instruction involves:

• Fetching the opcode from memory
• Dispatching to the handler
• Manipulating the value stack
• Updating reference counts

Modern CPUs optimise well for straight-line native code. Interpreter overhead typically adds 100–1000 ns per Python-level operation relative to equivalent C.

The Real Cost Distribution

Typical CPU-bound Python program breakdown:

  Actual computation:          ████░░░░░░░░░░░░░░░░  20%
  Dynamic dispatch overhead:   ████████░░░░░░░░░░░░  40%
  Object allocation/GC:        ██████░░░░░░░░░░░░░░  30%
  Interpreter loop overhead:   ██░░░░░░░░░░░░░░░░░░  10%

When you use NumPy or Cython, you eliminate the 80% overhead and keep the 20% actual work.

The Python Performance Stack

Understanding where your code sits in the stack tells you which optimisation direction is available:

┌─────────────────────────────────────────────────────────────────┐
│                    Your Application Code                        │
│              (pure Python loops, business logic)                │
│                   Typical speed: 1x baseline                    │
├─────────────────────────────────────────────────────────────────┤
│                     Python C Extensions                         │
│          (NumPy, pandas, scikit-learn, PIL, lxml, ...)          │
│              Speed relative to baseline: 10–100x                │
│    The extension releases the GIL, uses SIMD, avoids dispatch   │
├─────────────────────────────────────────────────────────────────┤
│               NumPy Vectorisation + Broadcasting                │
│       (express loops as array operations, BLAS/LAPACK)          │
│              Speed relative to baseline: 10–200x                │
│     Often the simplest optimisation available for numerical code │
├─────────────────────────────────────────────────────────────────┤
│                    Cython Compiled Code                         │
│         (.pyx → .c → .so, static types, typed memoryviews)      │
│              Speed relative to baseline: 50–500x                │
│         Full control over memory layout, GIL release possible   │
├─────────────────────────────────────────────────────────────────┤
│                    Numba JIT Compilation                        │
│         (LLVM IR, @njit, @vectorize, @cuda.jit for GPU)         │
│              Speed relative to baseline: 50–500x                │
│     No C compilation, GPU support, but restricted type system   │
├─────────────────────────────────────────────────────────────────┤
│                    C / Fortran / Rust                           │
│     (write the extension yourself, call via ctypes/cffi/PyO3)   │
│              Speed relative to baseline: 100–1000x              │
│           Maximum control, maximum complexity, rare need        │
├─────────────────────────────────────────────────────────────────┤
│                        Hardware                                 │
│         CPU cores, SIMD (AVX2/AVX-512), GPU (CUDA/ROCm)        │
│          Memory bandwidth, cache hierarchy, NUMA topology        │
└─────────────────────────────────────────────────────────────────┘

Each layer is an option. The correct layer depends on where your bottleneck actually is. This is why profiling comes first - always.

When Performance Matters (and When It Doesn't)

Here is an uncomfortable truth: most Python performance problems are not Python performance problems.

Before touching a profiler, classify your bottleneck:

Is the service slow?
        │
        ▼
  ┌─────────────────┐
  │ What are users  │
  │ actually waiting│
  │ for?            │
  └────────┬────────┘
           │
    ┌──────┴──────┐
    │             │
    ▼             ▼
 Database       Python
 query          CPU
    │             │
    ▼             ▼
Add an index   Profile
first          the code

The Premature Optimisation Tax

Knuth's rule is widely quoted and widely ignored: "We should forget about small efficiencies, say about 97% of the time: premature optimization is the root of all evil."

The tax you pay for premature optimisation:

• Readability loss: Cython .pyx files are less readable than Python
• Maintenance burden: optimised code is harder to change
• Wasted time: the function you optimised was 2% of runtime
• Opportunity cost: that time could have fixed a real user problem

Optimise when:

1. You have profiled and identified a specific bottleneck
2. The bottleneck accounts for ≥ 10% of total runtime
3. The expected speedup is meaningful to users or business metrics
4. You have a benchmark that demonstrates the improvement

Do not optimise when:

1. You "feel" something might be slow
2. The total runtime is already acceptable
3. The code runs once during startup (not in the hot path)
4. An architectural change would eliminate the problem entirely (better algorithm, caching, etc.)

The Measurement-First Discipline

Every professional performance engineer follows the same sequence:

1. ESTABLISH BASELINE
   ↓ Define the metric: wall time, p99 latency, memory usage, throughput
   ↓ Run a reproducible benchmark - same data, same environment, multiple runs

2. PROFILE
   ↓ Find the 20% of code consuming 80% of runtime
   ↓ Use cpu profiler + memory profiler - they reveal different problems

3. HYPOTHESISE
   ↓ "This function is slow because it creates 10M temporary objects"
   ↓ Form a specific, falsifiable hypothesis

4. CHANGE ONE THING
   ↓ Change only the suspected bottleneck
   ↓ Do not refactor while optimising

5. MEASURE AGAIN
   ↓ Compare against baseline
   ↓ If not improved → wrong hypothesis, go back to step 2

6. DOCUMENT
   ↓ Record what you changed and why
   ↓ Keep the benchmark as a regression test

This sounds obvious. In practice, engineers skip to step 4 and are confused when performance gets worse.

Module Lesson Map

Lesson	Title	Core Tool(s)	Typical Speedup
01	Profiling Python Applications	cProfile, py-spy, memory_profiler	N/A (measurement)
02	Cython and Native Extensions	Cython, ctypes, cffi	10–500x
03	Numba JIT Compilation	Numba @njit, @vectorize, @cuda.jit	10–500x
04	Python Memory Optimisation	slots, tracemalloc, mmap, gc	2–10x memory
05	Async Performance Patterns	uvloop, asyncio, connection pools	2–10x I/O
06	Python at Scale	multiprocessing, Ray, Dask, Celery	N × CPU cores

Performance Engineering Mindset

Before diving into the technical content, internalise these principles. They will save you more time than any specific technique.

Principle 1: Measure Before You Move

Your intuition about where performance problems are is wrong approximately 80% of the time. This is not a personal failing - it is a universal property of complex systems. Profilers exist precisely because human intuition fails at this task.

Principle 2: Understand the Problem Class

Problem Class	Symptom	Wrong Fix	Right Fix
CPU-bound	100% CPU, slow computation	Add more threads	Numba / Cython / C ext
I/O-bound	Low CPU, high wait time	More processes	asyncio / connection pool
Memory-bound	High RSS, OOM kills, GC pauses	More RAM	slots, generators
Algorithmic	O(n²) where O(n log n) exists	Any language trick	Fix the algorithm
Network-bound	High latency, not high CPU	Faster Python	Caching, CDN, batching
Database-bound	Slow queries, pool exhaustion	Python optimisation	Indexes, query tuning

Principle 3: The 80/20 Rule of Hot Paths

In virtually every real production system, 80% of CPU time is spent in 20% of the codebase. Your job is to identify that 20% before writing a single line of optimised code. Once you have found the hot path:

• If it is an algorithmic problem (O(n²) loop, redundant work) - fix the algorithm first
• If it is a Python-level loop over numbers - consider Numba or Cython
• If it is a Python-level loop over objects - rethink the data structure
• If it is already calling a C extension - investigate if you are calling it inefficiently

Principle 4: Benchmark Honestly

A benchmark that does not match production is worse than no benchmark - it gives false confidence. Benchmark requirements:

import timeit
import statistics

def benchmark(func, args, n_warmup=3, n_runs=20):
    """
    Honest benchmark: warm up JIT/caches, take multiple samples,
    report median (not mean - means are skewed by outliers).
    """
    # Warm up
    for _ in range(n_warmup):
        func(*args)

    # Collect samples
    times = []
    for _ in range(n_runs):
        start = time.perf_counter()
        func(*args)
        times.append(time.perf_counter() - start)

    return {
        "median": statistics.median(times),
        "mean": statistics.mean(times),
        "stdev": statistics.stdev(times),
        "min": min(times),
        "max": max(times),
        "p95": sorted(times)[int(0.95 * n_runs)],
    }

Always report median for latency benchmarks. Mean is distorted by outliers. Always report p95 or p99 for latency-sensitive services - the average being fast while p99 is terrible is a production incident waiting to happen.

Principle 5: Do Not Confuse Micro-Benchmarks with Real Performance

# This benchmark shows that list comprehensions are 30% faster than for loops
# for building a list of 1000 integers. It tells you almost nothing about
# whether your application is slow.

# Fast:
result = [x * 2 for x in range(1000)]

# Slower:
result = []
for x in range(1000):
    result.append(x * 2)

The 30% difference is real. But if this loop runs once per HTTP request and your database query takes 45ms, fixing this buys you 0.001ms. Profile first. Always.

Principle 6: Verify in Production Conditions

Laboratory benchmarks miss:

• Memory pressure from other processes
• CPU throttling under sustained load
• Cache effects from real data distributions
• GC pressure from the rest of the application
• I/O scheduling competition

The gold standard is A/B testing: deploy the optimised version to 5% of traffic, compare p50/p99/p999 latency and error rate, promote if better.

The Python Performance Ecosystem

Performance Engineering Tools
│
├── CPU Profiling
│   ├── cProfile        - deterministic, built-in, good for development
│   ├── line_profiler   - line-by-line timing, @profile decorator
│   ├── py-spy          - sampling, zero overhead, safe in production
│   ├── pyinstrument    - sampling, beautiful output, development
│   └── Austin          - frame stack sampler, SVG flamegraphs
│
├── Memory Profiling
│   ├── tracemalloc     - built-in, allocation tracking, snapshot diffs
│   ├── memory_profiler - line-by-line memory, @profile decorator
│   ├── objgraph        - object reference graphs, leak detection
│   └── guppy3/heapy    - heap analysis
│
├── Benchmarking
│   ├── timeit          - built-in, correct micro-benchmarks
│   ├── pytest-benchmark - benchmark suite integration
│   └── perf            - Linux performance counters (CPU cache misses, etc.)
│
├── Compilation / JIT
│   ├── Cython          - .pyx → C → .so, static typing
│   ├── Numba           - LLVM JIT, @njit, GPU support
│   ├── PyPy            - alternative interpreter, tracing JIT
│   └── mypyc           - mypy → C extension compiler
│
├── Parallelism
│   ├── multiprocessing - process pool, shared memory (stdlib)
│   ├── concurrent.futures - cleaner API over threads/processes
│   ├── Ray             - distributed actors, remote functions
│   ├── Dask            - lazy arrays/dataframes, task graphs
│   └── Celery          - task queues, retry logic, backends
│
└── Async I/O
    ├── asyncio         - built-in event loop
    ├── uvloop          - fast libuv-backed event loop
    ├── aiohttp         - async HTTP client/server
    └── httpx           - sync + async HTTP client

A Real-World Performance Story

This is a composite of real production incidents.

The Setup: A FastAPI service processes incoming documents. Each request tokenises the text, runs a similarity search, and returns the top 5 matching documents. At low traffic it runs fine. At 500 req/s the p99 latency spikes to 8 seconds and the service starts OOM-killing.

The Wrong Diagnosis: The team assumes the similarity search is slow (it uses a loop over 50,000 vectors). They spend two days porting it to Cython.

The Actual Profile:

Profile of handle_request() - 500 req/s, p99 = 8s

ncalls    tottime   cumtime   function
1000      0.003 s   0.003 s   tokenise()
1000      0.021 s   0.021 s   similarity_search()
1000      0.001 s   0.001 s   format_response()
1000      6.971 s   6.971 s   json.loads(request.body)  ← !!

The request body was 2MB of JSON (a long document). json.loads on 2MB JSON takes 7ms. At 500 req/s on a single worker, this is 3.5 seconds of CPU time per second - impossible to keep up with. The similarity search they optimised for two days was 21ms total.

The Fix: Switch to orjson (4x faster than stdlib json) and compress the request payload. Total time: 3 hours.

The Lesson: Never guess. Profile first.

Module Learning Path

Each lesson builds on the previous:

Lesson 01: Profiling  ──→  establishes measurement discipline
     │
     ▼
Lesson 02: Cython     ──→  CPU-bound loops, numerical code
     │
     ▼
Lesson 03: Numba      ──→  numerical loops, GPU acceleration
     │
     ▼
Lesson 04: Memory     ──→  reduce allocations, fix GC pressure
     │
     ▼
Lesson 05: Async      ──→  I/O-bound services, connection pools
     │
     ▼
Lesson 06: Scale      ──→  distribute across cores and machines

You can read lessons out of order if you have a specific problem. The prerequisite is Lesson 01 - you should always profile before applying any technique from lessons 02–06.

Quick Reference: Which Tool for Which Problem

This table condenses the module into a decision guide. Return to it after completing each lesson.

Symptom	First Check	Primary Fix
Slow numerical loops	line_profiler	NumPy vectorisation or Numba @njit
High CPU, complex non-numerical code	cProfile	Algorithm change or Cython
High memory usage	tracemalloc	__slots__, generators, mmap
Memory growing over time (leak)	objgraph	Fix circular refs, use weakref
Slow I/O service, low CPU	py-spy + logs	asyncio + connection pools
Single-core CPU saturation	cProfile	multiprocessing or Ray
Data too large for RAM	memory_profiler	Dask or mmap
Background jobs blocking request handler	logs / tracing	Celery or asyncio background tasks
Slow JSON parsing	cProfile	orjson or msgspec
Slow regex	line_profiler	Precompile regex, or re2

Summary

Python's performance characteristics are predictable once you understand the model:

1. The GIL prevents CPU parallelism from threads - use multiprocessing for CPU-bound parallel work
2. Dynamic dispatch adds 50–200ns per operation - eliminate it with static types (Cython) or JIT (Numba)
3. Object overhead inflates memory by 3–10x versus C arrays - use NumPy arrays or __slots__
4. Interpreter overhead costs ~100ns per bytecode instruction - batch work into C extension calls

The tools in this module address each of these costs specifically. The discipline that ties them together is measurement first: profile, identify, hypothesise, change one thing, measure again.

By the end of Module 1 you will have the full toolkit - measurement tools, compilation techniques, memory management strategies, async patterns, and distributed computing primitives - to make production Python code run as fast as the problem requires.

Next: Lesson 01 - Profiling Python Applications - because you cannot fix what you have not measured.