EngineersofAI LogoCoursesBlogResearch LabAI LettersThe LabCode BankInteractive 3DKodrEarnestJobs
Skip to main content

Import Hooks and the Import System - Intercepting Module Loading

Reading time: ~25 minutes | Level: Advanced

Predict what happens when you run this code:

import sys

class DeprecationFinder:
    DEPRECATED = {
        "old_utils": "new_utils",
    }

    def find_module(self, fullname, path=None):
        if fullname in self.DEPRECATED:
            return self

    def load_module(self, fullname):
        replacement = self.DEPRECATED[fullname]
        import warnings
        warnings.warn(
            f"Module '{fullname}' is deprecated. Use '{replacement}' instead.",
            DeprecationWarning,
            stacklevel=2,
        )
        __import__(replacement)
        sys.modules[fullname] = sys.modules[replacement]
        return sys.modules[fullname]

sys.meta_path.insert(0, DeprecationFinder())

import old_utils  # What happens here?
Click to reveal the answer

When import old_utils executes, Python consults sys.meta_path finders. DeprecationFinder.find_module returns self for "old_utils", claiming it can load the module. Python then calls load_module("old_utils"), which issues a DeprecationWarning, imports new_utils as the actual module, and installs it in sys.modules under both names. From this point, old_utils and new_utils are the same module object. This is a real pattern used for backwards-compatible module renames.

(Note: this uses the legacy finder/loader API. The modern approach uses MetaPathFinder.find_spec, covered below.)

What You Will Learn​

  • The complete import flow: sys.modules cache, sys.meta_path finders, sys.path entries, and the fallback chain
  • The MetaPathFinder and Loader protocols, and how ModuleSpec ties them together
  • How to write a custom finder that intercepts imports and loads from non-standard sources
  • How to implement lazy imports that defer module loading until first attribute access
  • How pytest rewrites assert statements using an import hook that transforms the AST
  • Why importing can execute arbitrary code, how __init__.py works, and how to fix circular imports
  • How to use importlib.metadata for reading package metadata and discovering plugins via entry points

Prerequisites​

  • Understanding of Python modules and packages (__init__.py, relative imports)
  • Familiarity with sys.path and how Python locates modules
  • Basic knowledge of Abstract Syntax Trees (helpful for Part 5, but not required)
  • Comfort with metaclasses and __init_subclass__ from previous lessons

Part 1 - How Python Imports Work​

When you write import foo, Python executes a specific search protocol. Understanding this protocol is the foundation for everything else in this lesson.

The Import Flow​

Step 1: The sys.modules Cache​

Every successfully imported module is cached in sys.modules, a dict mapping module names to module objects:

import sys
import json

print("json" in sys.modules)       # True
print(sys.modules["json"])          # <module 'json' from '...'>
print(json is sys.modules["json"])  # True - same object

# You can even manipulate the cache:
import json as j1
sys.modules["json"] = type(sys)("json")  # replace with empty module
import json as j2
print(j1 is j2)  # False - j2 is the replacement
danger

Modifying sys.modules is powerful but dangerous. Removing a module from the cache forces a re-import on next import statement, but objects that already reference the old module will still use it. This is a common source of subtle bugs in testing and hot-reloading systems.

Step 2: sys.meta_path Finders​

If the module is not cached, Python iterates through sys.meta_path - a list of finder objects:

import sys
for finder in sys.meta_path:
    print(type(finder).__name__)
# BuiltinImporter       - handles built-in modules (sys, builtins)
# FrozenImporter         - handles frozen modules (used in executables)
# PathFinder             - handles filesystem-based imports via sys.path

Each finder has a find_spec(fullname, path, target) method. The first finder to return a ModuleSpec (instead of None) wins.

Step 3: sys.path and the PathFinder​

The PathFinder (the last entry in sys.meta_path) searches sys.path - a list of directories and zip files:

import sys
for p in sys.path[:5]:
    print(p)
# '' (current directory)
# /usr/lib/python3.12
# /usr/lib/python3.12/lib-dynload
# /home/user/.local/lib/python3.12/site-packages
# ...

For each entry in sys.path, PathFinder consults sys.path_hooks - a list of callables that can return path entry finders for specific path types (directories, zip files, etc.).

Part 2 - Finders and Loaders​

The modern import system (PEP 302, refined in PEP 451) uses two key abstractions:

MetaPathFinder​

from importlib.abc import MetaPathFinder
from importlib.machinery import ModuleSpec

class MyFinder(MetaPathFinder):
    def find_spec(self, fullname, path, target=None):
        """
        Called for every import.

        Parameters:
            fullname: the fully qualified module name (e.g., "foo.bar")
            path:     the parent package's __path__ (None for top-level)
            target:   the module object being reloaded (None for fresh import)

        Returns:
            ModuleSpec if this finder can handle the import, else None.
        """
        return None  # default: cannot handle this import

Loader​

from importlib.abc import Loader

class MyLoader(Loader):
    def create_module(self, spec):
        """Return a module object, or None for default semantics."""
        return None  # use default module creation

    def exec_module(self, module):
        """Execute the module code in the module's namespace."""
        module.greeting = "Hello from MyLoader!"

ModuleSpec​

ModuleSpec ties the finder and loader together:

from importlib.machinery import ModuleSpec

spec = ModuleSpec(
    name="my_module",
    loader=MyLoader(),
    origin="<custom>",    # where the module comes from
    is_package=False,
)

The complete relationship:

Part 3 - Writing a Custom Finder​

Here is a practical example: a finder that loads modules from a dictionary of source code strings. This is useful for testing, embedded systems, or serving code from a database.

import sys
import types
from importlib.abc import MetaPathFinder, Loader
from importlib.machinery import ModuleSpec

class DictSourceLoader(Loader):
    """Loads module source code from a dictionary."""

    def __init__(self, source_dict):
        self.source_dict = source_dict

    def create_module(self, spec):
        return None  # use default module creation

    def exec_module(self, module):
        source = self.source_dict[module.__name__]
        code = compile(source, f"<dict:{module.__name__}>", "exec")
        exec(code, module.__dict__)


class DictSourceFinder(MetaPathFinder):
    """Finder that intercepts imports for modules stored in a dict."""

    def __init__(self, source_dict):
        self.source_dict = source_dict
        self.loader = DictSourceLoader(source_dict)

    def find_spec(self, fullname, path, target=None):
        if fullname in self.source_dict:
            return ModuleSpec(
                name=fullname,
                loader=self.loader,
                origin=f"<dict:{fullname}>",
            )
        return None


# --- Usage ---

# Define modules as source strings (could come from a database, API, etc.)
virtual_modules = {
    "greetings": """
def hello(name):
    return f"Hello, {name}!"

def goodbye(name):
    return f"Goodbye, {name}!"

LANGUAGE = "en"
""",
    "math_extras": """
def factorial(n):
    if n <= 1:
        return 1
    return n * factorial(n - 1)

PI_APPROX = 355 / 113
""",
}

# Install the finder
finder = DictSourceFinder(virtual_modules)
sys.meta_path.insert(0, finder)

# Now these modules are importable even though no files exist:
import greetings
print(greetings.hello("World"))    # Hello, World!
print(greetings.LANGUAGE)          # en

import math_extras
print(math_extras.factorial(10))   # 3628800
print(math_extras.PI_APPROX)      # 3.1415929203539825

# Clean up
sys.meta_path.remove(finder)
del sys.modules["greetings"]
del sys.modules["math_extras"]

A Namespace-Restricted Finder​

A more targeted pattern: a finder that only handles a specific namespace prefix, loading from a custom backend:

class NamespaceFinder(MetaPathFinder):
    """Only intercepts imports under 'plugins.*' namespace."""

    def __init__(self, plugin_registry):
        self.registry = plugin_registry

    def find_spec(self, fullname, path, target=None):
        if not fullname.startswith("plugins."):
            return None

        plugin_name = fullname[len("plugins."):]
        if plugin_name not in self.registry:
            return None

        return ModuleSpec(
            name=fullname,
            loader=PluginLoader(self.registry[plugin_name]),
            origin=f"<plugin:{plugin_name}>",
        )

class PluginLoader(Loader):
    def __init__(self, plugin_config):
        self.config = plugin_config

    def create_module(self, spec):
        return None

    def exec_module(self, module):
        # Load plugin from config (could be a file path, URL, etc.)
        module.name = self.config["name"]
        module.version = self.config["version"]
        # In reality, you would load and execute the plugin code here
tip

Always insert your custom finder at the beginning of sys.meta_path (using insert(0, ...)) if you want it to take priority over the default finders. Insert at the end if you want it to be a fallback.

Part 4 - Lazy Imports​

Lazy importing defers the actual module loading until the module is first accessed. This can dramatically improve application startup time for large codebases.

Using importlib.util.LazyLoader​

Python 3.4+ provides a built-in lazy loader:

import importlib.util
import sys

def lazy_import(name):
    """Import a module lazily - it is not loaded until first attribute access."""
    spec = importlib.util.find_spec(name)
    if spec is None:
        raise ModuleNotFoundError(f"No module named {name!r}")

    loader = importlib.util.LazyLoader(spec.loader)
    spec.loader = loader
    module = importlib.util.module_from_spec(spec)
    sys.modules[name] = module
    loader.exec_module(module)
    return module


# numpy is NOT loaded yet - just a lazy placeholder:
np = lazy_import("json")  # using json as example (always available)
print(type(np))  # <class 'module'> - looks like a module

# First attribute access triggers actual loading:
print(np.dumps({"key": "value"}))  # {"key": "value"}

Building a Custom Lazy Module​

For more control, you can build a lazy module proxy:

import importlib
import types

class LazyModule(types.ModuleType):
    """A module proxy that loads the real module on first attribute access."""

    def __init__(self, name):
        super().__init__(name)
        self._lazy_name = name
        self._lazy_loaded = False

    def _load(self):
        if not self._lazy_loaded:
            # Import the real module
            real_module = importlib.import_module(self._lazy_name)
            # Copy all attributes from the real module
            self.__dict__.update(real_module.__dict__)
            self._lazy_loaded = True

    def __getattr__(self, name):
        self._load()
        return self.__dict__[name]


def install_lazy(*module_names):
    """Install lazy proxies for the given module names."""
    for name in module_names:
        sys.modules[name] = LazyModule(name)


# Usage:
import sys
install_lazy("xml.etree.ElementTree", "csv", "sqlite3")

# None of these modules are loaded yet.
# They load on first attribute access:
import csv
reader = csv.reader(["a,b,c"])  # csv loads here
print(list(reader))  # [['a', 'b', 'c']]

When Lazy Imports Help​

Lazy imports are most valuable when:

  1. Large dependency trees: importing a module pulls in dozens of transitive dependencies
  2. CLI tools: most subcommands only use a subset of imports - loading all of them upfront wastes time
  3. Optional dependencies: a module may or may not be installed; lazy importing defers the ImportError
# Pattern: lazy optional dependency
def get_yaml():
    """Lazily import PyYAML - only fails if actually used."""
    try:
        import yaml
        return yaml
    except ImportError:
        raise ImportError(
            "PyYAML is required for YAML support. "
            "Install it with: pip install pyyaml"
        ) from None
note

Python 3.12+ includes experimental support for lazy imports via importlib. The Python core team is actively working on making lazy imports a first-class feature, potentially controlled by a command-line flag or __future__ import.

Part 5 - AST Transformation on Import​

This is one of the most powerful import hook patterns: modifying a module's source code (as an AST) before it is compiled. The most famous real-world example is pytest's assert rewriting.

How pytest Rewrites Assertions​

When you write:

def test_addition():
    result = add(2, 3)
    assert result == 5

A bare assert result == 5 would give you only AssertionError with no details. Pytest installs an import hook that transforms the AST before compilation:

# What pytest's import hook effectively does:

# Original AST:
# Assert(test=Compare(left=Name('result'), ops=[Eq()], comparators=[Num(5)]))

# Rewritten AST (simplified):
# if not result == 5:
#     raise AssertionError(
#         f"assert {result!r} == 5\n"
#         f"  where {result!r} = add(2, 3)"
#     )

Building a Simple AST-Transforming Import Hook​

Here is a working import hook that auto-instruments all function calls with logging:

import ast
import sys
import types
import importlib.abc
import importlib.machinery
from pathlib import Path

class LoggingTransformer(ast.NodeTransformer):
    """AST transformer that adds print statements at the start of every function."""

    def visit_FunctionDef(self, node):
        # Create: print(f"CALL: {function_name}()")
        log_stmt = ast.parse(
            f'print(f"CALL: {node.name}()")'
        ).body[0]

        # Preserve line numbers
        ast.copy_location(log_stmt, node)
        ast.fix_missing_locations(log_stmt)

        # Insert at the beginning of the function body
        node.body.insert(0, log_stmt)

        # Continue visiting child nodes
        self.generic_visit(node)
        return node


class ASTTransformLoader(importlib.abc.SourceLoader):
    """Loader that applies AST transformations before compilation."""

    def __init__(self, original_loader, transformer):
        self.original_loader = original_loader
        self.transformer = transformer

    def get_filename(self, fullname):
        return self.original_loader.get_filename(fullname)

    def get_data(self, path):
        return self.original_loader.get_data(path)

    def source_to_code(self, data, path, *, _optimize=-1):
        """Override: parse source, transform AST, then compile."""
        source = data.decode("utf-8") if isinstance(data, bytes) else data
        tree = ast.parse(source, filename=path)
        tree = self.transformer.visit(tree)
        ast.fix_missing_locations(tree)
        return compile(tree, path, "exec", dont_inherit=True,
                       optimize=_optimize)


class ASTTransformFinder(importlib.abc.MetaPathFinder):
    """Finder that wraps the default loader with AST transformation."""

    def __init__(self, transformer, target_packages=None):
        self.transformer = transformer
        self.target_packages = target_packages or set()

    def find_spec(self, fullname, path, target=None):
        # Only transform specified packages
        if not any(fullname.startswith(pkg) for pkg in self.target_packages):
            return None

        # Find the spec using the normal mechanism
        # (temporarily remove ourselves to avoid recursion)
        sys.meta_path.remove(self)
        try:
            spec = importlib.util.find_spec(fullname)
        finally:
            sys.meta_path.insert(0, self)

        if spec is None or spec.loader is None:
            return None

        # Wrap the loader with our AST transformer
        spec.loader = ASTTransformLoader(spec.loader, self.transformer)
        return spec

Usage (conceptual - requires actual module files to transform):

# Install the hook for a specific package:
transformer = LoggingTransformer()
finder = ASTTransformFinder(transformer, target_packages={"myapp"})
sys.meta_path.insert(0, finder)

# Now, importing any module under "myapp" will have logging injected:
# import myapp.handlers  # every function gets a print("CALL: ...") at start
danger

AST transformation import hooks are powerful but create significant debugging challenges. The source code you see in your editor is not what actually runs. Use them sparingly and document them heavily. Pytest limits its assertion rewriting to test files only - it does not rewrite application code.

Part 6 - Import-Time Side Effects​

Why Importing Executes Code​

When Python imports a module, it executes the entire module body. This means importing can trigger arbitrary side effects:

# mymodule.py
print("Module is being imported!")  # runs on import

DB_CONNECTION = connect_to_database()  # runs on import

class MyClass:
    pass  # class body executes, __init_subclass__ fires

register_signal_handlers()  # runs on import

This is by design - module-level code initializes the module. But it creates problems:

__init__.py and Package Imports​

When you import a package, Python executes its __init__.py:

# mypackage/__init__.py
from .core import Engine       # imports core module
from .utils import helpers     # imports utils module
print("Package initialized")  # side effect

Heavy __init__.py files that import everything are a major cause of slow startup:

# BAD: __init__.py that imports everything
from .models import User, Product, Order, ...
from .views import UserView, ProductView, ...
from .serializers import ...

# GOOD: __init__.py that imports minimally
# Let users import what they need explicitly:
# from mypackage.models import User

Circular Imports​

Circular imports are the most common import-related bug. They happen when two modules depend on each other:

# a.py
from b import b_function

def a_function():
    return "a"

# b.py
from a import a_function  # ImportError or AttributeError!

def b_function():
    return "b"

When a.py starts importing, it runs from b import b_function. This triggers importing b.py, which tries from a import a_function. But a.py has not finished executing - a_function does not exist yet.

Strategies to fix circular imports:

1. Move the import inside the function:

# b.py
def b_function():
    from a import a_function  # deferred - only imported when called
    return a_function()

2. Import the module, not the name:

# b.py
import a  # works - the module object exists even if incomplete

def b_function():
    return a.a_function()  # accessed at call time, not import time

3. Restructure to eliminate the cycle:

tip

The cleanest solution is almost always restructuring. If a and b both need something from each other, that shared functionality belongs in a third module common that both can import without cycles.

Part 7 - importlib.metadata - Package Metadata and Plugin Discovery​

importlib.metadata (Python 3.8+) provides access to installed package metadata and entry points - the standard mechanism for plugin discovery.

Reading Package Metadata​

from importlib.metadata import metadata, version, packages_distributions

# Get package version:
print(version("pip"))      # e.g., 24.0
print(version("setuptools"))  # e.g., 69.5.1

# Get full metadata:
pip_meta = metadata("pip")
print(pip_meta["Summary"])  # The PyPA recommended tool for installing Python packages.
print(pip_meta["Author"])
print(pip_meta["License"])

Entry Points for Plugin Discovery​

Entry points are the standard mechanism for Python plugin systems. A package declares entry points in its pyproject.toml or setup.cfg, and other code discovers them at runtime:

# In a plugin package's pyproject.toml:
[project.entry-points."myapp.plugins"]
json_handler = "myapp_json:JsonPlugin"
xml_handler = "myapp_xml:XmlPlugin"
from importlib.metadata import entry_points

# Discover all plugins registered under "myapp.plugins":
plugins = entry_points(group="myapp.plugins")

for ep in plugins:
    print(f"Plugin: {ep.name}")
    print(f"  Value: {ep.value}")
    print(f"  Module: {ep.module}")

    # Load the plugin class:
    plugin_cls = ep.load()
    plugin = plugin_cls()
    plugin.handle(data)

Building a Plugin System with Entry Points​

A complete pattern for entry-point-based plugin discovery:

from importlib.metadata import entry_points
from abc import ABC, abstractmethod

class PluginInterface(ABC):
    """All plugins must implement this interface."""

    @abstractmethod
    def name(self) -> str:
        ...

    @abstractmethod
    def execute(self, data: dict) -> dict:
        ...


class PluginManager:
    """Discovers and manages plugins via entry points."""

    ENTRY_POINT_GROUP = "myapp.processors"

    def __init__(self):
        self._plugins: dict[str, PluginInterface] = {}
        self._discover()

    def _discover(self):
        """Load all installed plugins."""
        eps = entry_points(group=self.ENTRY_POINT_GROUP)
        for ep in eps:
            try:
                plugin_cls = ep.load()
                if not issubclass(plugin_cls, PluginInterface):
                    print(f"Warning: {ep.name} does not implement PluginInterface")
                    continue
                plugin = plugin_cls()
                self._plugins[plugin.name()] = plugin
                print(f"Loaded plugin: {plugin.name()}")
            except Exception as e:
                print(f"Failed to load plugin {ep.name}: {e}")

    def get(self, name: str) -> PluginInterface:
        if name not in self._plugins:
            raise KeyError(
                f"Plugin {name!r} not found. "
                f"Available: {list(self._plugins.keys())}"
            )
        return self._plugins[name]

    def all_plugins(self) -> list[PluginInterface]:
        return list(self._plugins.values())

    def execute_all(self, data: dict) -> list[dict]:
        """Run all plugins on the data and return results."""
        results = []
        for name, plugin in self._plugins.items():
            try:
                result = plugin.execute(data)
                results.append({"plugin": name, "result": result})
            except Exception as e:
                results.append({"plugin": name, "error": str(e)})
        return results

This is the same pattern used by:

  • pytest for discovering plugins (pytest11 entry point group)
  • Flask for CLI commands
  • setuptools for console_scripts
  • pip for build backends
note

Entry points are the Pythonic way to build plugin systems for distributed packages. For plugins within a single codebase, __init_subclass__ (covered in a previous lesson) is simpler and does not require package metadata.

Key Takeaways​

  • Python's import flow: check sys.modules cache, then iterate sys.meta_path finders, then search sys.path
  • Custom finders implement find_spec() and return a ModuleSpec with a Loader
  • Loaders implement create_module() and exec_module() to create and populate module objects
  • Lazy imports defer module loading until first attribute access - critical for startup performance
  • AST transformation hooks (like pytest's assert rewriting) modify source code before compilation
  • Importing a module executes its entire body - be aware of side effects, especially in __init__.py
  • Circular imports are caused by mutual dependencies at import time - fix by deferring imports, importing modules instead of names, or restructuring
  • importlib.metadata.entry_points() is the standard mechanism for plugin discovery across installed packages

Graded Practice Challenges​

Level 1 - Predict the Output​

Question 1:

import sys

# Assume no module named "phantom" exists anywhere
print("phantom" in sys.modules)

try:
    import phantom
except ModuleNotFoundError:
    pass

print("phantom" in sys.modules)
Answer
False
False

A failed import does not add the module to sys.modules. The cache only stores successfully imported modules. This is important - if the first import fails, a subsequent import phantom will retry the search (it does not cache the failure).

Question 2:

import sys
import types

# Manually inject a module into sys.modules
fake = types.ModuleType("fake_module")
fake.value = 42
fake.greet = lambda name: f"Hi, {name}"
sys.modules["fake_module"] = fake

import fake_module
print(fake_module.value)
print(fake_module.greet("World"))
print(type(fake_module))
Answer
42
Hi, World
<class 'module'>

sys.modules is checked before any finders are consulted. By inserting a module object directly into sys.modules, import fake_module finds it immediately and returns it. No file system search occurs. This is how mock modules, virtual modules, and module aliases work.

Question 3:

import sys

original_meta_path_len = len(sys.meta_path)

class NullFinder:
    def find_spec(self, fullname, path, target=None):
        if fullname == "blocked_module":
            raise ModuleNotFoundError(f"{fullname} is blocked!")
        return None

sys.meta_path.insert(0, NullFinder())

try:
    import json  # already in sys.modules from earlier
    print("json imported successfully")
except ModuleNotFoundError:
    print("json blocked")

try:
    import blocked_module
    print("blocked_module imported successfully")
except ModuleNotFoundError as e:
    print(f"Error: {e}")

sys.meta_path.pop(0)
Answer
json imported successfully
Error: blocked_module is blocked!

json was already cached in sys.modules from a previous import, so the finders are never consulted - it succeeds regardless of the NullFinder. blocked_module is not cached, so Python checks sys.meta_path. NullFinder.find_spec raises ModuleNotFoundError, which propagates up.

Level 2 - Debug Challenge​

This import hook is supposed to create "virtual config modules" that expose environment variables as module attributes, but it has a bug:

import sys
import os
import types
from importlib.abc import MetaPathFinder, Loader
from importlib.machinery import ModuleSpec

class EnvLoader(Loader):
    def __init__(self, prefix):
        self.prefix = prefix

    def create_module(self, spec):
        return types.ModuleType(spec.name)

    def exec_module(self, module):
        prefix = self.prefix.upper() + "_"
        for key, value in os.environ.items():
            if key.startswith(prefix):
                attr_name = key[len(prefix):].lower()
                setattr(module, attr_name, value)

class EnvFinder(MetaPathFinder):
    def find_spec(self, fullname, path, target=None):
        if fullname.startswith("config."):
            section = fullname.split(".")[-1]
            return ModuleSpec(
                name=fullname,
                loader=EnvLoader(section),
            )

sys.meta_path.insert(0, EnvFinder())

# Set some env vars for testing:
os.environ["DATABASE_HOST"] = "localhost"
os.environ["DATABASE_PORT"] = "5432"

# This should work:
import config.database
print(config.database.host)  # Expected: localhost
print(config.database.port)  # Expected: 5432
Hint

What happens when you import config.database? Python first needs to import the config package. Does config exist? What does the finder return for fullname="config"?

Solution

The bug: when Python processes import config.database, it first imports config as a package. The finder's check fullname.startswith("config.") does NOT match "config" (no dot), so find_spec returns None for the parent. Python then raises ModuleNotFoundError: No module named 'config'.

Fix: also handle the parent package:

class EnvFinder(MetaPathFinder):
    def find_spec(self, fullname, path, target=None):
        if fullname == "config":
            # Create the parent package
            spec = ModuleSpec(
                name="config",
                loader=PackageLoader(),
                is_package=True,
            )
            spec.submodule_search_locations = []
            return spec
        if fullname.startswith("config."):
            section = fullname.split(".")[-1]
            return ModuleSpec(
                name=fullname,
                loader=EnvLoader(section),
            )

class PackageLoader(Loader):
    def create_module(self, spec):
        return types.ModuleType(spec.name)

    def exec_module(self, module):
        module.__path__ = []  # mark as package

Level 3 - Design Challenge​

Build a "module versioning" import hook system with these requirements:

  1. A VersionedFinder that intercepts imports like import mylib_v2 and maps them to actual modules with version selection
  2. A registry where you register module versions: register_version("mylib", 2, "path/to/mylib_v2.py")
  3. import mylib_v2 loads the registered v2 source, import mylib_v1 loads v1
  4. The loaded module should have a __version__ attribute set automatically
  5. If the requested version is not registered, raise ModuleNotFoundError with a helpful message listing available versions
  6. A set_default_version("mylib", 2) that makes import mylib load v2

Your implementation should handle the finder, loader, and version registry. Test with at least two versions of a mock module.

Solution Sketch
import sys
import types
from importlib.abc import MetaPathFinder, Loader
from importlib.machinery import ModuleSpec
import re

class VersionRegistry:
    def __init__(self):
        self._versions = {}  # {module_name: {version: source}}
        self._defaults = {}  # {module_name: version}

    def register(self, name, version, source):
        self._versions.setdefault(name, {})[version] = source

    def set_default(self, name, version):
        if name not in self._versions or version not in self._versions[name]:
            raise ValueError(f"No version {version} registered for {name}")
        self._defaults[name] = version

    def get(self, name, version):
        versions = self._versions.get(name, {})
        if version not in versions:
            available = list(versions.keys())
            raise ModuleNotFoundError(
                f"Version {version} of '{name}' not found. "
                f"Available: {available}"
            )
        return versions[version]

    def get_default(self, name):
        if name in self._defaults:
            return self._defaults[name], self._versions[name][self._defaults[name]]
        return None, None

    def has_module(self, name):
        return name in self._versions


registry = VersionRegistry()

class VersionedLoader(Loader):
    def __init__(self, source, version):
        self.source = source
        self.version = version

    def create_module(self, spec):
        return None

    def exec_module(self, module):
        module.__version__ = self.version
        code = compile(self.source, f"<{module.__name__}>", "exec")
        exec(code, module.__dict__)


class VersionedFinder(MetaPathFinder):
    VERSION_PATTERN = re.compile(r"^(.+)_v(\d+)$")

    def find_spec(self, fullname, path, target=None):
        # Check for versioned import: mylib_v2
        match = self.VERSION_PATTERN.match(fullname)
        if match:
            name, ver = match.group(1), int(match.group(2))
            if registry.has_module(name):
                source = registry.get(name, ver)
                return ModuleSpec(
                    name=fullname,
                    loader=VersionedLoader(source, ver),
                )

        # Check for default version: mylib
        if registry.has_module(fullname):
            ver, source = registry.get_default(fullname)
            if ver is not None:
                return ModuleSpec(
                    name=fullname,
                    loader=VersionedLoader(source, ver),
                )
        return None


# Install
sys.meta_path.insert(0, VersionedFinder())

# Register versions
registry.register("mylib", 1, "def greet(): return 'Hello v1'")
registry.register("mylib", 2, "def greet(): return 'Hello v2'\ndef new_feature(): return 'v2 only'")
registry.set_default("mylib", 2)

# Test
import mylib_v1
print(mylib_v1.greet())       # Hello v1
print(mylib_v1.__version__)   # 1

import mylib_v2
print(mylib_v2.greet())       # Hello v2
print(mylib_v2.new_feature()) # v2 only
print(mylib_v2.__version__)   # 2

import mylib
print(mylib.greet())           # Hello v2 (default)
print(mylib.__version__)       # 2

What's Next​

You have now completed the core metaprogramming lessons - metaclasses, descriptors, __init_subclass__, __set_name__, dynamic class creation, and import hooks. In the module project, you will combine these techniques to build a custom ORM core from scratch, applying everything from descriptors to metaclasses to plugin discovery.

© 2026 EngineersOfAI. All rights reserved.