EngineersofAI LogoCoursesBlogResearch LabAI LettersThe LabCode BankInteractive 3DKodrEarnestJobs
Skip to main content

sys and inspect - Runtime Introspection at Engineering Depth

Reading time: ~35 minutes | Level: Intermediate → Engineering

Before reading further, predict what this program prints:

import inspect

def greet(name: str, greeting: str = "Hello") -> str:
    return f"{greeting}, {name}!"

sig = inspect.signature(greet)
for name, param in sig.parameters.items():
    print(f"{name}: default={param.default}, annotation={param.annotation}")
Show Answer
name: default=<class 'inspect._empty'>, annotation=<class 'str'>
greeting: default=Hello, annotation=<class 'str'>

Parameters without defaults report inspect.Parameter.empty - a sentinel class, not None. None is a valid default value (e.g., def f(x=None)), so inspect uses a distinct sentinel to mean "no default was specified at all."

This is exactly how FastAPI reads your function signature at startup. When you write:

@app.get("/users/{user_id}")
def get_user(user_id: int, db: Session = Depends(get_db)):
    ...

FastAPI calls inspect.signature(get_user), iterates over sig.parameters, and checks each parameter's annotation and default. If the default is Depends(...), FastAPI registers it as a dependency. If the annotation is a Pydantic model, FastAPI builds a request body validator. All of this happens at import time - before a single request arrives.

sys and inspect are the two modules that give Python programs access to their own runtime internals. sys exposes the interpreter state; inspect exposes the source code and call structure of live objects. Every serious framework you use - pytest, FastAPI, Django, click - relies on both. This lesson teaches you to use them as a framework author would.

What You Will Learn​

  • sys.argv, sys.path, sys.modules and how they control program execution
  • sys.getrefcount(), sys.getsizeof(), sys.setrecursionlimit()
  • sys.exc_info(): the current exception triple used by logging and error handlers
  • sys._getframe() and sys.settrace() / sys.setprofile(): how debuggers and coverage tools work
  • sys.stdin, sys.stdout, sys.stderr and redirecting them
  • inspect.signature() and all five parameter kinds
  • inspect.getmembers(), inspect.getsource(), inspect.getsourcelines()
  • inspect.stack() and inspect.currentframe()
  • How FastAPI, pytest, and click use inspect as their core mechanism

Prerequisites​

  • Lesson 05 (Decorators) - many inspect patterns involve decorated functions
  • Lesson 06 (Closures and Free Variables) - sys._getframe() exposes frame locals/globals
  • Python type annotations (PEP 484) - inspect.signature() reads __annotations__

Part 1 - The sys Module​

sys.argv: Command-Line Arguments​

import sys

# sys.argv[0] is always the script name
# sys.argv[1:] are the arguments passed on the command line

# script.py
if len(sys.argv) < 2:
    print(f"Usage: {sys.argv[0]} <name>")
    sys.exit(1)

name = sys.argv[1]
print(f"Hello, {name}!")
$ python script.py Alice
Hello, Alice!

$ python script.py
Usage: script.py <name>

sys.argv is a plain list[str]. For anything beyond trivial argument parsing, use argparse (standard library) or click (third party) - both read sys.argv internally.

sys.path: The Module Search Path​

import sys

# sys.path is the list of directories Python searches for modules
print(sys.path)
# ['', '/usr/lib/python312.zip', '/usr/lib/python3.12', ...]
# '' means the current working directory

# You can append to sys.path at runtime to import from non-standard locations
import os
sys.path.insert(0, "/opt/internal-packages")   # highest priority - searched first
import internal_tool   # now findable

# More robust: use site.addsitedir() or PYTHONPATH environment variable
warning

Mutating sys.path at runtime is fragile - the modification applies only to the current process and is not visible to subprocesses or other workers. Prefer installing packages into a virtual environment, or setting PYTHONPATH before the interpreter starts.

sys.modules: The Module Cache​

sys.modules is a dict mapping module names to their loaded module objects. Every successful import stores the result here. A second import of the same name hits this cache instead of executing the module again.

import sys
import json

# json is already in the cache
print("json" in sys.modules)         # True
print(sys.modules["json"] is json)   # True - same object

# You can inspect what is loaded
stdlib_modules = [name for name in sys.modules if "." not in name]
print(len(stdlib_modules))   # varies by environment - typically 50–100 at startup
note

sys.modules is the import cache. You can manually insert objects into it to make a module "importable" without a file on disk. Testing frameworks and mocking libraries use this trick: sys.modules["requests"] = mock_requests makes import requests return the mock. This is how unittest.mock.patch works at the module level.

sys.getrefcount(): Reference Count of an Object​

import sys

x = []
print(sys.getrefcount(x))   # 2: one for 'x', one for the argument to getrefcount

y = x
print(sys.getrefcount(x))   # 3: 'x', 'y', and the argument

del y
print(sys.getrefcount(x))   # 2 again

The count is always at least 1 higher than you expect because the act of passing the object to getrefcount creates a temporary reference.

sys.setrecursionlimit() and sys.getrecursionlimit()​

Python's default recursion limit is 1000. Deep recursive algorithms - tree traversals, parsers, certain graph algorithms - can exceed it.

import sys

print(sys.getrecursionlimit())   # 1000

# Increase for algorithms that require deeper recursion
sys.setrecursionlimit(10_000)

def depth_first(node, depth=0):
    if node is None or depth > 9_000:
        return
    depth_first(node.left, depth + 1)
    depth_first(node.right, depth + 1)
warning

Setting sys.setrecursionlimit to a very high value (e.g., 1 000 000) can cause a C stack overflow - a segmentation fault, not a Python RecursionError - because each Python frame consumes C stack space. The safe upper limit depends on your OS stack size (typically 1–8 MB). Prefer iterative algorithms with an explicit stack for very deep recursion.

sys.exc_info(): The Current Exception Triple​

Inside an except block, sys.exc_info() returns the currently-handled exception as (type, value, traceback).

import sys
import logging

def run_with_logging(func, *args, **kwargs):
    try:
        return func(*args, **kwargs)
    except Exception:
        exc_type, exc_value, exc_tb = sys.exc_info()
        logging.error(
            "Exception in %s: %s: %s",
            func.__name__,
            exc_type.__name__,
            exc_value,
        )
        raise   # re-raise the original exception

# This pattern is used in Django's middleware and Celery's task runner
# to log exceptions with full context before allowing them to propagate

Outside an except block, sys.exc_info() returns (None, None, None).

sys._getframe(): Accessing the Call Stack​

sys._getframe(depth) returns the frame object at depth levels up the call stack. Frame 0 is the caller, frame 1 is the caller's caller, etc.

import sys

def who_called_me() -> str:
    frame = sys._getframe(1)   # 1 = my caller's frame
    return f"{frame.f_code.co_filename}:{frame.f_lineno} in {frame.f_code.co_name}"

def application_function():
    print(who_called_me())

application_function()
# example.py:10 in application_function

Frame attributes:

  • f_code.co_filename - source file
  • f_code.co_name - function or method name
  • f_lineno - current line number
  • f_locals - local variables at that frame (a snapshot dict)
  • f_globals - global namespace of the frame's module
  • f_back - parent frame (one level up)
warning

sys._getframe() is a CPython implementation detail - the leading underscore signals it is not part of the public API and may not exist on other Python implementations (PyPy, Jython). Use inspect.currentframe() for slightly more portable code, or inspect.stack() for full call stack information.

sys.settrace() and sys.setprofile(): How Debuggers Work​

sys.settrace(func) installs a trace function called on every line execution, function call, and function return. sys.setprofile(func) is similar but fires only on calls and returns - lower overhead.

import sys

call_log = []

def trace_calls(frame, event, arg):
    if event == "call":
        call_log.append(frame.f_code.co_name)
    return trace_calls   # must return itself to trace nested calls

sys.settrace(trace_calls)

def add(x, y):
    return x + y

def multiply(x, y):
    return x * y

result = add(1, 2) + multiply(3, 4)

sys.settrace(None)   # disable tracing

print(call_log)   # ['add', 'multiply']

The trace function signature is (frame, event, arg) where event is one of:

  • "call" - entering a function (arg is None)
  • "line" - about to execute a line (arg is None)
  • "return" - about to return from a function (arg is the return value)
  • "exception" - an exception has been raised (arg is (type, value, traceback))
danger

sys.settrace fires on every line of Python execution. The overhead is severe - 10–100x slowdown is typical. Tools like pdb, coverage.py, and IDE debuggers use sys.settrace internally but always disable it when not actively debugging. Never leave sys.settrace active in production code.

sys.stdin, sys.stdout, sys.stderr: Redirecting I/O​

import sys
import io

# Capture stdout for testing
captured = io.StringIO()
sys.stdout = captured

print("This goes to the StringIO buffer, not the terminal")

sys.stdout = sys.__stdout__   # restore original stdout
output = captured.getvalue()
print(f"Captured: {repr(output)}")
# Captured: 'This goes to the StringIO buffer, not the terminal\n'

sys.__stdout__ (double underscore on both sides) is always the original stdout, even after reassignment. This is how capsys in pytest captures print output during tests without modifying the test code.

Part 2 - The inspect Module​

inspect.signature(): Full Parameter Introspection​

inspect.signature() returns a Signature object containing all parameters, their kinds, defaults, and annotations.

Parameter Kinds​

import inspect

def complex_func(
    pos_only: int,           # POSITIONAL_ONLY
    /,
    regular: str,            # POSITIONAL_OR_KEYWORD
    *args: float,            # VAR_POSITIONAL
    kw_only: bool = True,    # KEYWORD_ONLY
    **kwargs: str,           # VAR_KEYWORD
) -> None:
    pass

sig = inspect.signature(complex_func)
for name, param in sig.parameters.items():
    print(f"{name:12} kind={param.kind.name:25} default={param.default}")
pos_only     kind=POSITIONAL_ONLY          default=<class 'inspect._empty'>
regular      kind=POSITIONAL_OR_KEYWORD    default=<class 'inspect._empty'>
args         kind=VAR_POSITIONAL           default=<class 'inspect._empty'>
kw_only      kind=KEYWORD_ONLY             default=True
kwargs       kind=VAR_KEYWORD              default=<class 'inspect._empty'>

Binding Arguments with signature.bind()​

import inspect

def create_user(name: str, age: int, role: str = "viewer") -> dict:
    return {"name": name, "age": age, "role": role}

sig = inspect.signature(create_user)

# Validate args before calling - raises TypeError with a clear message on failure
try:
    bound = sig.bind("Alice", 30)            # missing 'role' is fine - has default
    bound.apply_defaults()
    print(bound.arguments)
    # OrderedDict([('name', 'Alice'), ('age', 30), ('role', 'viewer')])
except TypeError as e:
    print(f"Bad arguments: {e}")

# Missing required argument
try:
    sig.bind("Alice")   # missing 'age'
except TypeError as e:
    print(f"Bad arguments: {e}")
    # Bad arguments: missing a required argument: 'age'
tip

Use inspect.signature(func).bind(*args, **kwargs) to validate arguments before calling a function dynamically. This gives you the same error message Python would give - "missing a required argument: 'x'" - without actually calling the function. FastAPI uses this pattern to validate that all dependency parameters can be satisfied before registering a route.

How FastAPI Uses inspect.signature​

inspect.getmembers(), isfunction(), isclass(), ismethod()​

import inspect

class MyService:
    version = "1.0"

    def process(self, data: dict) -> dict:
        return data

    @classmethod
    def from_config(cls, config: dict) -> "MyService":
        return cls()

    @staticmethod
    def validate(data: dict) -> bool:
        return bool(data)

# Get all members, filtered by type
methods = inspect.getmembers(MyService, predicate=inspect.isfunction)
print([name for name, _ in methods])
# ['from_config', 'process', 'validate']

# ismethod vs isfunction on instances
svc = MyService()
print(inspect.ismethod(svc.process))       # True  - bound method
print(inspect.isfunction(svc.process))     # False - bound methods are not functions
print(inspect.isfunction(MyService.process))  # True - unbound, it is a function

# isclass
print(inspect.isclass(MyService))    # True
print(inspect.isclass(svc))          # False

inspect.getsource(), getsourcefile(), getsourcelines()​

import inspect

def compute_tax(income: float, rate: float = 0.2) -> float:
    """Compute income tax at the given rate."""
    if income <= 0:
        return 0.0
    return income * rate

# Get the full source as a string
src = inspect.getsource(compute_tax)
print(src)
# def compute_tax(income: float, rate: float = 0.2) -> float:
#     """Compute income tax at the given rate."""
#     ...

# Get the source file path
print(inspect.getsourcefile(compute_tax))
# /path/to/example.py

# Get source lines and starting line number
lines, start_line = inspect.getsourcelines(compute_tax)
print(f"Defined at line {start_line}")
print("".join(lines))
warning

inspect.getsource() fails for built-in functions and functions defined in C extensions. Calling inspect.getsource(len) raises OSError: could not get source code. It also fails for functions defined in the interactive REPL. Always wrap in a try/except when the source is not guaranteed to be available.

inspect.stack() and inspect.currentframe()​

inspect.stack() returns the full call stack as a list of FrameInfo named tuples, from the current frame outward.

import inspect

def level_three():
    stack = inspect.stack()
    for i, frame_info in enumerate(stack[:4]):
        print(f"[{i}] {frame_info.filename}:{frame_info.lineno} in {frame_info.function}")
        if frame_info.code_context:
            print(f"    {frame_info.code_context[0].strip()}")

def level_two():
    level_three()

def level_one():
    level_two()

level_one()
[0] example.py:4  in level_three
    stack = inspect.stack()
[1] example.py:11 in level_two
    level_three()
[2] example.py:14 in level_one
    level_two()
[3] example.py:17 in <module>
    level_one()

FrameInfo fields:

  • frame - the actual frame object (use with care - keeping frame references causes memory leaks)
  • filename - source file path
  • lineno - current line number
  • function - function name
  • code_context - list of source lines around the current line
  • index - index within code_context of the current line

inspect.getdoc() and inspect.cleandoc()​

import inspect

class DataProcessor:
    def transform(self, data):
        """
        Transform raw data into the canonical format.

        Strips whitespace, normalises unicode, and validates structure.
        Raises ValueError if data does not match the expected schema.

        Args:
            data: Raw input dict from the API layer.

        Returns:
            Normalised dict ready for database insertion.
        """
        pass

# inspect.getdoc normalises indentation automatically
print(inspect.getdoc(DataProcessor.transform))
# Transform raw data into the canonical format.
#
# Strips whitespace, normalises unicode, and validates structure.
# ...

# inspect.cleandoc does the same normalisation on a raw string
raw = """
        First line.
        Second line.
        Third line.
        """
print(inspect.cleandoc(raw))
# First line.
# Second line.
# Third line.

Part 3 - Real-World Framework Usage​

pytest: Fixture Injection via Parameter Names​

pytest reads the names of your test function's parameters to decide which fixtures to inject:

# pytest reads inspect.signature(test_function)
# and matches parameter names to registered fixtures

def test_user_creation(db_session, mock_email_service, tmp_path):
    # pytest sees: parameter 'db_session' → inject db_session fixture
    #              parameter 'mock_email_service' → inject that fixture
    #              parameter 'tmp_path' → inject built-in tmp_path fixture
    user = User.create(session=db_session, name="Alice")
    mock_email_service.assert_called_once_with(user.email)

Under the hood, pytest does roughly:

import inspect

def resolve_fixtures(test_func, fixture_registry: dict):
    sig = inspect.signature(test_func)
    kwargs = {}
    for param_name, param in sig.parameters.items():
        if param_name in fixture_registry:
            kwargs[param_name] = fixture_registry[param_name]()
        elif param.default is inspect.Parameter.empty:
            raise pytest.FixtureError(f"No fixture named '{param_name}'")
    return test_func(**kwargs)

click: CLI Arguments from Function Signatures​

click uses inspect.signature() to map CLI arguments to function parameters:

import click

@click.command()
@click.option("--name", required=True)
@click.option("--count", default=1, type=int)
def hello(name: str, count: int):
    for _ in range(count):
        click.echo(f"Hello, {name}!")

# click reads inspect.signature(hello) to understand
# that 'name' and 'count' are the function's parameters,
# then maps CLI flags to them by name

dataclasses: Field Inspection​

The dataclasses module uses inspect to read class annotations and build __init__, __repr__, and __eq__:

import inspect
import dataclasses

@dataclasses.dataclass
class Point:
    x: float
    y: float
    label: str = ""

# What dataclasses does internally:
fields = {
    name: annotation
    for name, annotation in inspect.get_annotations(Point).items()
}
print(fields)
# {'x': <class 'float'>, 'y': <class 'float'>, 'label': <class 'str'>}

# inspect.fields() on a dataclass gives you the full Field objects
for field in dataclasses.fields(Point):
    print(f"{field.name}: {field.type}, default={field.default}")

Graded Practice​

Level 1 - Predict the Output​

import inspect

def outer(x: int) -> int:
    def inner(y: int = 10) -> int:
        return x + y
    return inner

sig_outer = inspect.signature(outer)
sig_inner = inspect.signature(outer(5))

print(len(sig_outer.parameters))
print(len(sig_inner.parameters))

param = sig_inner.parameters["y"]
print(param.default)
print(param.default is inspect.Parameter.empty)
Show Answer
1
1
10
False

outer has one parameter (x). outer(5) returns inner, which has one parameter (y with default 10). param.default is 10 (an integer, not inspect.Parameter.empty), so the last line prints False.

Level 2 - Debug This Code​

This decorator is supposed to log the name and arguments of every function call. It works but breaks inspect.signature() on decorated functions. Fix it.

import functools

def log_calls(func):
    def wrapper(*args, **kwargs):
        print(f"Calling {func.__name__} args={args} kwargs={kwargs}")
        return func(*args, **kwargs)
    return wrapper

@log_calls
def create_order(user_id: int, product: str, quantity: int = 1) -> dict:
    return {"user": user_id, "product": product, "qty": quantity}

import inspect
sig = inspect.signature(create_order)
print(sig)   # prints (*args, **kwargs) - wrong!
Show Answer

functools.wraps is missing. Without it, wrapper.__wrapped__ is not set and inspect.signature() sees the generic (*args, **kwargs) signature of wrapper instead of the original function's signature.

Fixed version:

import functools

def log_calls(func):
    @functools.wraps(func)          # <-- add this
    def wrapper(*args, **kwargs):
        print(f"Calling {func.__name__} args={args} kwargs={kwargs}")
        return func(*args, **kwargs)
    return wrapper

@log_calls
def create_order(user_id: int, product: str, quantity: int = 1) -> dict:
    return {"user": user_id, "product": product, "qty": quantity}

import inspect
sig = inspect.signature(create_order)
print(sig)
# (user_id: int, product: str, quantity: int = 1) -> dict  - correct

functools.wraps sets wrapper.__wrapped__ = func. inspect.signature() follows __wrapped__ chains to find the innermost original function's signature. This is also why FastAPI's dependency injection works correctly even when your route handler functions are decorated.

Level 3 - Design Challenge​

Design a lightweight dependency injection container inspired by pytest's fixture system, using inspect.signature. It should:

  1. Allow registering "providers" - functions that create objects (similar to pytest fixtures)
  2. When a registered provider or any other function is called through the container, automatically resolve its parameter names by looking them up in the registry
  3. Support transitive resolution (a provider can itself depend on other registered providers)
  4. Raise a clear KeyError with the missing dependency name if a parameter cannot be resolved

Write the Container class with register(name, factory) and call(func) methods, then demonstrate it with a three-level dependency chain.

Show Answer
import inspect
from typing import Callable, Any

class Container:
    def __init__(self):
        self._registry: dict[str, Callable] = {}
        self._cache: dict[str, Any] = {}

    def register(self, name: str, factory: Callable) -> None:
        """Register a factory function under a name."""
        self._registry[name] = factory

    def resolve(self, name: str) -> Any:
        """Resolve a named dependency, caching the result (singleton behaviour)."""
        if name not in self._cache:
            if name not in self._registry:
                raise KeyError(f"No dependency registered for '{name}'")
            factory = self._registry[name]
            self._cache[name] = self.call(factory)
        return self._cache[name]

    def call(self, func: Callable) -> Any:
        """Call func, injecting all its parameters from the registry."""
        sig = inspect.signature(func)
        kwargs = {}
        for param_name, param in sig.parameters.items():
            if param_name in self._registry:
                kwargs[param_name] = self.resolve(param_name)
            elif param.default is not inspect.Parameter.empty:
                kwargs[param_name] = param.default
            else:
                raise KeyError(
                    f"Cannot inject parameter '{param_name}' into {func.__name__}: "
                    f"not registered and has no default"
                )
        return func(**kwargs)

# --- Demonstration ---

container = Container()

# Level 1: no dependencies
container.register("config", lambda: {"db_url": "sqlite:///app.db", "debug": True})

# Level 2: depends on config
def make_db_connection(config: dict):
    return f"Connection to {config['db_url']}"

container.register("db", make_db_connection)

# Level 3: depends on db and config
def make_user_repository(db, config: dict):
    return {"connection": db, "debug": config["debug"]}

container.register("user_repo", make_user_repository)

# Resolve the full chain
repo = container.resolve("user_repo")
print(repo)
# {'connection': 'Connection to sqlite:///app.db', 'debug': True}

# Call an ad-hoc function that uses the container
def run_report(user_repo, config: dict) -> str:
    return f"Report for {user_repo['connection']} (debug={config['debug']})"

result = container.call(run_report)
print(result)
# Report for Connection to sqlite:///app.db (debug=True)

This is the core of how pytest fixtures, FastAPI Depends, and Python IoC frameworks like injector work. inspect.signature provides the parameter names; the framework maps names to registered providers.

Key Takeaways​

  • sys.modules is the module cache: every successfully imported module is stored here and the second import of the same name returns the cached object without re-executing the module file.
  • sys.argv is a plain list[str]; sys.path is the list of directories searched for imports; both are mutable at runtime.
  • sys.exc_info() returns (type, value, traceback) for the currently-handled exception - the standard pattern for re-raising with logging in middleware and error handlers.
  • sys._getframe(n) gives direct access to the call stack at depth n, exposing local variables, source file, and line number. Used by logging formatters to report "called from" information.
  • sys.settrace fires on every line and is how pdb, coverage.py, and IDE debuggers work. It is too slow for production - always disable when not actively debugging.
  • inspect.signature() is the foundation of Python's modern framework ecosystem. pytest, FastAPI, click, and dataclasses all use it to map parameter names to injectable values.
  • The five Parameter.kind values - POSITIONAL_ONLY, POSITIONAL_OR_KEYWORD, VAR_POSITIONAL, KEYWORD_ONLY, VAR_KEYWORD - correspond directly to the five syntactic forms in a function definition.
  • inspect.signature(func).bind(*args, **kwargs) validates arguments without calling the function, producing the same error message Python would produce.
  • inspect.getsource() fails on built-ins and C extensions - always guard with try/except OSError.
  • inspect.stack() returns the full call stack with source context - useful for debugging and for framework code that needs to know the call site.
© 2026 EngineersOfAI. All rights reserved.