Project 01 - Bytecode Visualizer

Estimated time: 5–7 hours core | Level: Intermediate

Before reading the requirements, consider this question: when Python executes return n * factorial(n - 1), it does not execute that as a single operation. It emits a sequence of stack-manipulation instructions - loads, a binary multiply, a call - that the CPython interpreter executes one at a time. How many bytecode instructions does that single line produce? Run import dis; dis.dis(lambda n: n * (n - 1)) and count. Understanding the answer before you start is the foundation of this project.

Learning Objectives

By the time this project is complete, you will have practiced:

• Parsing Python source into an AST using the ast module and traversing it programmatically
• Using compile() to produce a code object and dis.get_instructions() to extract structured instruction data
• Understanding what each common bytecode instruction does - LOAD_FAST, LOAD_GLOBAL, CALL, BINARY_OP, RETURN_VALUE - and writing human-readable explanations for them
• Simulating CPython's value stack manually - pushing and popping values as each instruction executes
• Detecting performance-relevant patterns in bytecode (LOAD_FAST vs LOAD_GLOBAL)
• Implementing a side-by-side diff of two bytecode sequences
• Exporting structured analysis data as JSON for downstream tooling

System Overview

You are building a single module visualizer.py that exports one class: BytecodeVisualizer. The class takes a source code string at construction time, compiles it, and exposes four analysis methods. The module has no external dependencies - standard library only. No rich, no colorama, no pygments.

from visualizer import BytecodeVisualizer

viz = BytecodeVisualizer("""
def factorial(n):
    if n <= 1:
        return 1
    return n * factorial(n - 1)
""")

viz.show_ast()          # pretty-print the AST
viz.show_bytecode()     # annotated dis output with explanations
viz.show_stack_trace()  # simulate value stack evolution step by step
viz.compare("""
def factorial(n):
    result = 1
    while n > 1:
        result *= n
        n -= 1
    return result
""")  # side-by-side bytecode comparison

Requirements

R1 - AST parsing and pretty-printing

Parse the source string into an AST using ast.parse(). show_ast() must print the AST in a readable tree format that shows each node's type and, where applicable, its value. For each node, include the node type name and the line number.

• Use ast.dump() with indent=2 as the baseline output, then augment it with node type annotations in the printed form.
• Alternatively, write a recursive visitor using ast.NodeVisitor that prints a custom indented tree - this earns higher marks for understanding.
• The output must be deterministic for the same source string.
• Line numbers must appear for all statement and expression nodes that carry them (node.lineno).

R2 - Bytecode disassembly

show_bytecode() must disassemble the compiled code object using dis.get_instructions(), which returns a sequence of dis.Instruction named tuples. Do not parse the string output of dis.dis() - use the structured API.

• For each function defined in the source, disassemble its code object separately (not just the module-level code object).
• Each instruction line must show: offset, opname, arg (numeric), argval (resolved value), and the human-readable explanation from R3.
• Visually separate each function's bytecode with a header showing the function name.

R3 - Human-readable instruction explanations

For each instruction, provide a one-line English explanation of what it does to the stack and why. This is not documentation copying - it is a description in terms of the current context.

The explanation table must cover at minimum:

Opname	Explanation
LOAD_FAST	Push local variable {argval} onto the stack
LOAD_GLOBAL	Push global variable {argval} onto the stack (slower than LOAD_FAST)
LOAD_CONST	Push constant {argval} onto the stack
STORE_FAST	Pop top of stack and store as local variable {argval}
BINARY_OP	Pop two values, apply {argval}, push result
COMPARE_OP	Pop two values, compare with {argval}, push bool result
POP_JUMP_IF_FALSE	Pop top of stack; jump to offset {arg} if it is False
JUMP_BACKWARD	Unconditional jump to offset {arg} (loop back-edge)
CALL	Call the callable with {arg} arguments from the stack
RETURN_VALUE	Pop top of stack and return it to the caller
RESUME	Internal interpreter checkpoint (no stack effect)

For any opname not in your table, output Instruction: {opname} as a fallback.

R4 - Value stack simulation

show_stack_trace() must simulate the CPython value stack step by step for the first function found in the source. After each instruction executes, print the current state of the stack.

• Represent stack values symbolically - use the argval of LOAD instructions as the symbolic name of the value pushed (e.g., pushing LOAD_CONST 1 shows [1] on the stack; pushing LOAD_FAST n shows [n] on the stack).
• For binary operations, pop two symbolic values and push a symbolic expression (e.g., n * factorial(n-1)).
• For CALL instructions, pop the callable and arguments, push a symbolic call result (e.g., factorial(n-1)).
• For RETURN_VALUE, show the value being returned and mark the simulation complete.
• You do not need to simulate jumps - for conditional jumps, note both possible outcomes and stop simulation at that point.

R5 - Performance pattern detection

During show_bytecode(), detect and highlight performance-relevant patterns:

• Any LOAD_GLOBAL instruction inside a loop (between a JUMP_BACKWARD target and a JUMP_BACKWARD instruction) must be marked with a [PERF] tag and a note: LOAD_GLOBAL inside loop - cache as local variable for ~20% speedup.
• Any function that contains zero LOAD_FAST instructions and only LOAD_GLOBAL instructions (i.e., uses no local variables) must emit a note after the bytecode output.
• Count and report the ratio of LOAD_FAST to LOAD_GLOBAL calls at the end of each function's disassembly.

R6 - Side-by-side comparison

compare(other_source) accepts a second source string, disassembles both, and prints their bytecode side by side with the same column width.

• Align the two sequences by line number within each function, padding with blank lines where one sequence is longer.
• Mark instructions that are present in one but not the other with a < or > indicator in a centre column.
• At the bottom of the comparison, print: instruction count for each, and which version has fewer instructions.

R7 - Instruction category counts

After each function's disassembly, print an instruction count table broken into categories:

Category	Opcodes included	Count
Loads	LOAD_FAST, LOAD_GLOBAL, LOAD_CONST, LOAD_DEREF	-
Stores	STORE_FAST, STORE_GLOBAL, STORE_DEREF	-
Jumps	POP_JUMP_IF_FALSE, POP_JUMP_IF_TRUE, JUMP_BACKWARD, JUMP_FORWARD	-
Calls	CALL, CALL_FUNCTION, CALL_METHOD	-
Returns	RETURN_VALUE, RETURN_CONST	-
Other	everything else	-

R8 - JSON export

Implement an export_json() method that returns a JSON-serialisable dict containing the full analysis: AST summary, per-function bytecode (list of instruction dicts), instruction category counts, and detected performance patterns. The dict must be serialisable with json.dumps() without a custom encoder.

data = viz.export_json()
import json
print(json.dumps(data, indent=2))

Starter Code Skeleton

import ast
import dis
import json
from dataclasses import dataclass, field
from typing import Optional


# ── Data Structures ────────────────────────────────────────────────────────────

@dataclass
class InstructionInfo:
    offset: int
    opname: str
    arg: Optional[int]
    argval: object
    explanation: str
    is_perf_warning: bool = False
    perf_note: str = ""


@dataclass
class FunctionAnalysis:
    name: str
    instructions: list[InstructionInfo] = field(default_factory=list)
    category_counts: dict[str, int] = field(default_factory=dict)
    perf_warnings: list[str] = field(default_factory=list)


# ── Explanation Table ──────────────────────────────────────────────────────────

INSTRUCTION_EXPLANATIONS = {
    "LOAD_FAST":         "Push local variable {argval} onto the stack",
    "LOAD_GLOBAL":       "Push global variable {argval} onto the stack (slower than LOAD_FAST)",
    "LOAD_CONST":        "Push constant {argval} onto the stack",
    "STORE_FAST":        "Pop top of stack and store as local variable {argval}",
    "BINARY_OP":         "Pop two values, apply {argval}, push result",
    "COMPARE_OP":        "Pop two values, compare with {argval}, push bool result",
    "POP_JUMP_IF_FALSE": "Pop top of stack; jump to offset {arg} if it is False",
    "JUMP_BACKWARD":     "Unconditional jump to offset {arg} (loop back-edge)",
    "CALL":              "Call the callable with {arg} arguments from the stack",
    "RETURN_VALUE":      "Pop top of stack and return it to the caller",
    "RETURN_CONST":      "Return constant {argval} to the caller",
    "RESUME":            "Internal interpreter checkpoint (no stack effect)",
}


def explain(instr: dis.Instruction) -> str:
    """Return a human-readable explanation for a dis.Instruction."""
    template = INSTRUCTION_EXPLANATIONS.get(instr.opname)
    if template is None:
        return f"Instruction: {instr.opname}"
    # TODO: format template with instr.argval and instr.arg
    pass


# ── Instruction Categories ─────────────────────────────────────────────────────

CATEGORIES = {
    "Loads":   {"LOAD_FAST", "LOAD_GLOBAL", "LOAD_CONST", "LOAD_DEREF"},
    "Stores":  {"STORE_FAST", "STORE_GLOBAL", "STORE_DEREF"},
    "Jumps":   {"POP_JUMP_IF_FALSE", "POP_JUMP_IF_TRUE", "JUMP_BACKWARD", "JUMP_FORWARD"},
    "Calls":   {"CALL", "CALL_FUNCTION", "CALL_METHOD"},
    "Returns": {"RETURN_VALUE", "RETURN_CONST"},
}


def categorise(opname: str) -> str:
    """Return the category name for a given opname."""
    for category, opcodes in CATEGORIES.items():
        if opname in opcodes:
            return category
    return "Other"


# ── Main Class ─────────────────────────────────────────────────────────────────

class BytecodeVisualizer:
    def __init__(self, source: str):
        self.source = source.strip()
        self._tree = ast.parse(self.source)
        self._code = compile(self.source, "<visualizer>", "exec")
        # TODO: extract per-function code objects from self._code.co_consts
        self._functions: dict[str, object] = {}  # name -> code object

    def _extract_functions(self) -> None:
        """Walk co_consts recursively to find nested code objects (functions)."""
        # TODO: iterate self._code.co_consts
        # TODO: if an item is a code object (types.CodeType), add it to self._functions keyed by co_name
        pass

    def show_ast(self) -> None:
        """Pretty-print the AST with node types and line numbers."""
        # TODO: use ast.dump(self._tree, indent=2) for a baseline
        # TODO: or implement a custom ast.NodeVisitor that prints an indented tree
        pass

    def show_bytecode(self) -> None:
        """Disassemble all functions and print annotated bytecode with explanations."""
        for name, code_obj in self._functions.items():
            print(f"\n{'='*60}")
            print(f"  Function: {name}")
            print(f"{'='*60}")
            analysis = self._analyse_function(code_obj)
            self._print_function_analysis(analysis)

    def _analyse_function(self, code_obj) -> FunctionAnalysis:
        """Build a FunctionAnalysis from a code object."""
        analysis = FunctionAnalysis(name=code_obj.co_name)
        instructions = list(dis.get_instructions(code_obj))

        # TODO: detect loop ranges (JUMP_BACKWARD targets) for PERF detection
        # TODO: for each instruction, build InstructionInfo with explanation
        # TODO: mark LOAD_GLOBAL inside loop as perf warning
        # TODO: count categories

        return analysis

    def _print_function_analysis(self, analysis: FunctionAnalysis) -> None:
        """Print formatted analysis for one function."""
        # TODO: print each InstructionInfo as a formatted line
        # TODO: highlight perf warnings with [PERF] tag
        # TODO: print category count table
        # TODO: print LOAD_FAST / LOAD_GLOBAL ratio
        pass

    def show_stack_trace(self) -> None:
        """Simulate value stack evolution for the first function in source."""
        # TODO: get first function's code object
        # TODO: iterate instructions, maintaining a list as the symbolic stack
        # TODO: for each instruction type, push/pop symbolic values
        # TODO: print stack state after each instruction
        pass

    def compare(self, other_source: str) -> None:
        """Print side-by-side bytecode comparison of self.source vs other_source."""
        other = BytecodeVisualizer(other_source)
        # TODO: for each function name present in both, zip their instructions
        # TODO: pad the shorter list with blank lines
        # TODO: print aligned columns with < > indicators
        # TODO: print instruction count summary at the bottom
        pass

    def export_json(self) -> dict:
        """Return a JSON-serialisable dict of the full analysis."""
        # TODO: build dict with keys: source, ast_summary, functions
        # TODO: each function entry: name, instructions (list of dicts), category_counts, perf_warnings
        # TODO: verify json.dumps(result) does not raise
        return {}


# ── Demonstration ──────────────────────────────────────────────────────────────

if __name__ == "__main__":
    recursive_source = """
def factorial(n):
    if n <= 1:
        return 1
    return n * factorial(n - 1)
"""

    iterative_source = """
def factorial(n):
    result = 1
    while n > 1:
        result *= n
        n -= 1
    return result
"""

    viz = BytecodeVisualizer(recursive_source)

    print("=" * 60)
    print("AST")
    print("=" * 60)
    viz.show_ast()

    print("\n" + "=" * 60)
    print("BYTECODE WITH EXPLANATIONS")
    print("=" * 60)
    viz.show_bytecode()

    print("\n" + "=" * 60)
    print("STACK TRACE SIMULATION")
    print("=" * 60)
    viz.show_stack_trace()

    print("\n" + "=" * 60)
    print("SIDE-BY-SIDE COMPARISON")
    print("=" * 60)
    viz.compare(iterative_source)

    print("\n" + "=" * 60)
    print("JSON EXPORT (first 20 lines)")
    print("=" * 60)
    data = viz.export_json()
    lines = json.dumps(data, indent=2).splitlines()
    print("\n".join(lines[:20]))
    print("...")

Expected Output

============================================================
AST
============================================================
Module (line -)
  FunctionDef: factorial (line 2)
    arguments
      arg: n
    If (line 3)
      Compare (line 3)
        Name: n
        LtE
        Constant: 1
      Return (line 4)
        Constant: 1
    Return (line 5)
      BinOp: Mult (line 5)
        Name: n
        Call: factorial (line 5)
          BinOp: Sub (line 5)
            Name: n
            Constant: 1

============================================================
BYTECODE WITH EXPLANATIONS
============================================================

============================================================
  Function: factorial
============================================================
  Offset  Opname               Arg   Argval              Explanation
  ------  -------------------  ----  ------------------  ----------------------------------------
       0  RESUME               0     0                   Internal interpreter checkpoint (no stack effect)
       2  LOAD_FAST            0     n                   Push local variable n onto the stack
       4  LOAD_CONST           1     1                   Push constant 1 onto the stack
       6  COMPARE_OP           1     <=                  Pop two values, compare with <=, push bool result
      10  POP_JUMP_IF_FALSE    4     14                  Pop top of stack; jump to offset 14 if it is False
      12  RETURN_CONST         1     1                   Return constant 1 to the caller
      14  LOAD_FAST            0     n                   Push local variable n onto the stack
      16  LOAD_GLOBAL          1     factorial           Push global variable factorial onto the stack (slower than LOAD_FAST)
      28  LOAD_FAST            0     n                   Push local variable n onto the stack
      30  LOAD_CONST           1     1                   Push constant 1 onto the stack
      32  BINARY_OP            10    -                   Pop two values, apply -, push result
      36  CALL                 1     1                   Call the callable with 1 arguments from the stack
      46  BINARY_OP            5     *                   Pop two values, apply *, push result
      50  RETURN_VALUE         0     None                Pop top of stack and return it to the caller

Instruction category counts:
  Loads   : 5
  Stores  : 0
  Jumps   : 1
  Calls   : 1
  Returns : 2
  Other   : 5

LOAD_FAST / LOAD_GLOBAL ratio: 3 / 1

============================================================
STACK TRACE SIMULATION
============================================================
Function: factorial

  Step  Offset  Opname               Stack after instruction
  ----  ------  -------------------  ----------------------------------------
     1       0  RESUME               []
     2       2  LOAD_FAST            [n]
     3       4  LOAD_CONST           [n, 1]
     4       6  COMPARE_OP           [(n <= 1)]
     5      10  POP_JUMP_IF_FALSE    []  -- if False, jump to 14; if True, continue
     6      12  RETURN_CONST         returns 1
  [simulation continues from offset 14 assuming jump taken]
     7      14  LOAD_FAST            [n]
     8      16  LOAD_GLOBAL          [n, factorial]
     9      28  LOAD_FAST            [n, factorial, n]
    10      30  LOAD_CONST           [n, factorial, n, 1]
    11      32  BINARY_OP            [n, factorial, (n - 1)]
    12      36  CALL                 [n, factorial(n-1)]
    13      46  BINARY_OP            [(n * factorial(n-1))]
    14      50  RETURN_VALUE         returns (n * factorial(n-1))

============================================================
SIDE-BY-SIDE COMPARISON
============================================================

Function: factorial

  Recursive                                  │   │  Iterative
  ─────────────────────────────────────────  │   │  ─────────────────────────────────────────
  RESUME                                     │ = │  RESUME
  LOAD_FAST n                                │ = │  LOAD_FAST n
  LOAD_CONST 1                               │ = │  LOAD_CONST 1
  COMPARE_OP <=                              │ = │  COMPARE_OP >
  POP_JUMP_IF_FALSE                          │ = │  POP_JUMP_IF_FALSE
  RETURN_CONST 1                             │ < │
                                             │ > │  STORE_FAST result
  LOAD_FAST n                                │ = │  LOAD_FAST n
  LOAD_GLOBAL factorial                      │ < │
  ...                                        │   │  ...

  Recursive: 14 instructions
  Iterative: 18 instructions
  Recursive version has fewer instructions (14 vs 18)

Note: exact offset values and minor formatting details will vary across Python versions. The structure and content must match.

Step-by-Step Hints

Hint 1 - Start with _extract_functions and verify it works before anything else. A code object's co_consts is a tuple that may contain other code objects - one per function defined at the top level of the module. To find them:

import types

for const in self._code.co_consts:
    if isinstance(const, types.CodeType):
        self._functions[const.co_name] = const

Call _extract_functions in __init__. Verify it finds your function by printing self._functions.keys() before building anything else.

Hint 2 - Use dis.get_instructions(), not dis.dis(). dis.dis() prints to stdout and returns None. dis.get_instructions() is the structured API - it returns an iterator of dis.Instruction named tuples with fields: opname, opcode, arg, argval, argrepr, offset, starts_line, is_jump_target. Always use get_instructions() and format the output yourself.

Hint 3 - Detecting loops for R5. A loop in bytecode has a JUMP_BACKWARD instruction that jumps to an earlier offset. Any LOAD_GLOBAL instruction whose offset is between the target of a JUMP_BACKWARD and the JUMP_BACKWARD itself is inside a loop. Pre-compute the set of loop ranges in one pass, then annotate in a second pass:

instructions = list(dis.get_instructions(code_obj))
loop_ranges = []
for instr in instructions:
    if instr.opname == "JUMP_BACKWARD":
        loop_ranges.append((instr.argval, instr.offset))
# instr.argval for JUMP_BACKWARD is the target offset

def in_loop(offset: int) -> bool:
    return any(start <= offset <= end for start, end in loop_ranges)

Hint 4 - Symbolic stack for show_stack_trace. Maintain a Python list as your symbolic stack. For each instruction:

stack = []
for instr in dis.get_instructions(code_obj):
    if instr.opname in ("LOAD_FAST", "LOAD_GLOBAL", "LOAD_CONST"):
        stack.append(repr(instr.argval))
    elif instr.opname == "BINARY_OP":
        right = stack.pop()
        left = stack.pop()
        stack.append(f"({left} {instr.argval} {right})")
    elif instr.opname == "CALL":
        n_args = instr.arg
        args = stack[-n_args:]  # last n_args items
        del stack[-n_args - 1:]  # remove callable + args
        stack.append(f"call_result")
    ...

This is a simplified model. Real CPython stack simulation is more involved - for this project, symbolic accuracy matters more than exact fidelity.

Hint 5 - Side-by-side comparison formatting. Use zip_longest from itertools to pair up instruction lines from the two functions:

from itertools import zip_longest

col_width = 42
for left, right in zip_longest(left_lines, right_lines, fillvalue=""):
    indicator = "=" if left.strip() == right.strip() else "<" if right == "" else ">" if left == "" else "~"
    print(f"  {left:<{col_width}} │ {indicator} │  {right}")

Hint 6 - JSON export. dis.Instruction is a named tuple, not directly JSON-serialisable. Convert each instruction to a plain dict:

def instruction_to_dict(instr: dis.Instruction) -> dict:
    return {
        "offset": instr.offset,
        "opname": instr.opname,
        "arg": instr.arg,
        "argval": str(instr.argval),  # str() handles non-serialisable argvals
        "argrepr": instr.argrepr,
    }

Internals Concepts Tested

Concept	Where it appears
ast.parse() and ast.NodeVisitor	R1 - AST pretty-printing
compile()	__init__ - produces the module code object
types.CodeType and co_consts	_extract_functions - locates nested function code objects
dis.get_instructions()	R2, R3, R4, R5, R6, R7 - structured bytecode access
CPython value stack model	R4 - stack simulation
Jump instruction analysis	R5 - loop detection
json.dumps()	R8 - export

Engineering Notes - Where dis Is Used in Production

Coverage measurement. coverage.py uses a combination of sys.settrace() and bytecode analysis to determine which lines have been executed. It analyses the compiled bytecode to understand the control flow graph - which lines can be reached from which - and uses that to report on untested branches. Understanding dis output is essential for understanding how coverage.py decides what counts as a covered branch.

Python optimisers. The peephole optimiser in CPython itself operates on bytecode. It removes dead code, folds constant expressions, and replaces slow sequences with faster equivalents. Third-party tools like codon and pypy's JIT compiler both start by analysing the same bytecode CPython produces. When you run dis.dis(lambda: 1 + 1) and see LOAD_CONST 2 rather than LOAD_CONST 1; LOAD_CONST 1; BINARY_OP +, you are seeing the peephole optimiser's output.

Security scanners. Tools like bandit and dlint analyse Python source (via AST) and bytecode to detect security vulnerabilities. Bytecode analysis allows scanners to detect obfuscated calls or dynamic constructs that AST-level analysis would miss. The pattern of iterating dis.get_instructions() and checking for specific opnames is exactly how these tools detect calls to dangerous functions.

Jupyter and IPython. IPython's %%timeit magic patches the bytecode of the cell being timed. It inserts a loop around the cell's code object and measures execution time. Understanding how compile() and code objects work is a prerequisite for understanding how these tools modify code without touching the source text.

Extension Challenges

Extension 1 - Control flow graph Build a show_cfg() method that draws the control flow graph of a function as text. Each basic block (sequence of instructions with no jumps into or out of the middle) is a node. Edges are drawn for unconditional jumps, conditional true/false branches, and fall-through. Use ASCII box-drawing characters.

Extension 2 - Decompiler Write a decompile() method that attempts to reconstruct Python source from bytecode without using the original source. This is non-trivial - handle LOAD/STORE pairs for assignments, COMPARE_OP + POP_JUMP for if-statements, and JUMP_BACKWARD for while loops. Do not use any decompiler libraries.

Extension 3 - Python version diff Run the same source through compile() and dis.get_instructions() but target different Python versions by using ast.parse() with feature_version argument where applicable. Compare how bytecode changes across versions (e.g., how CALL_FUNCTION became CALL in Python 3.11, or how LOAD_ATTR changed in 3.12).

Extension 4 - Bytecode patcher Use types.CodeType constructor arguments to create a modified copy of a function's code object with one constant replaced (e.g., change a hardcoded threshold value). Verify the patched function behaves differently. This is the basis of how mocking libraries patch functions without access to the source.

Extension 5 - Live trace comparison Use sys.settrace() alongside your bytecode visualizer to record which instructions are actually executed during a real call (e.g., factorial(5)). Overlay the execution trace on your bytecode output - highlight instructions that were executed and show how many times each was hit. This is how coverage.py produces line-level hit counts.