Formatting and Tooling - Automate Code Quality

Reading time: ~25 minutes | Level: Foundation → Engineering

$ git commit -m "add payment processing"
black....................................................................Failed
- hook id: black
- files were modified by this hook

reformatted src/payments.py

Your commit was blocked. Not by a teammate, not by a CI system \text{---} by a pre-commit hook running Black on your local machine. You did not format the file. Black did it in 200 milliseconds, automatically, correctly. You stage the changes and commit again. It passes.

This is what a professional Python toolchain looks like. No formatting debates. No "you have a trailing space on line 47" review comments. No style inconsistency across 20 engineers working on the same codebase. The tools enforce the rules; engineers focus on the logic.

This lesson covers the complete Python formatting and quality toolchain: Black, isort, flake8, mypy, and pre-commit hooks. By the end, you will have a working .pre-commit-config.yaml, a configured pyproject.toml, and the mental model to maintain this setup across any project.

What You Will Learn

Why the "tabs vs spaces" wars ended and what ended them
Black: the opinionated formatter that changed Python's ecosystem
isort: automated import sorting that integrates with Black
flake8: linting for style violations and real bugs
mypy: static type checking to catch type errors before runtime
pre-commit hooks: enforcing quality on every commit, locally
pyproject.toml: the modern configuration hub for all tools
A complete GitHub Actions CI pipeline running the full toolchain

Prerequisites

Familiarity with Python projects, modules, and virtual environments
Ability to run commands in a terminal
Basic understanding of git commits and staging
Some familiarity with type annotations is helpful for the mypy section

The End of the Tabs vs Spaces Wars

In 2018, a survey by Stack Overflow found that developers who used spaces earned more than developers who used tabs. This absurd finding went viral because it touched a nerve: the formatting wars were real, ongoing, and consuming real engineering time.

Teams argued about:

Tabs vs spaces (and which width)
Single vs double quotes
Line length (79 vs 99 vs 120 characters)
Trailing commas in multi-line structures
Whether to put function arguments on the same line or separate lines

Code reviews had comments like "please use double quotes consistently" and "this line is 84 characters, PEP 8 says 79." These were not engineering discussions. They were noise that slowed down the review of actual logic.

In 2018, Łukasz Langa released Black. The tagline: "The Uncompromising Code Formatter." Black does not ask you what style you prefer. It formats your code its way, and its way is non-negotiable. There are essentially no configuration options for style (only line-length and a few practical toggles).

The reaction was initially divisive. Then teams adopted Black and discovered that formatting debates disappeared overnight. Not because everyone agreed with every Black choice, but because the conversation shifted: instead of "should we use single or double quotes?" the conversation became "should we use Black?" And once you adopt Black, the first question never needs to be asked again.

Today, Black is the dominant Python formatter. It is used by CPython itself, by Django, by NumPy, by Pandas, by FastAPI, and by virtually every major Python open source project. In professional environments, "use Black" is the baseline expectation.

Black: The Opinionated Formatter

Installation

pip install black
# Or with Python version specifier
pip install "black[d]"  # includes blackd, the formatting daemon

Basic Usage

# Format a single file
black src/mymodule.py

# Format all Python files in a directory
black src/

# Check without modifying (useful in CI)
black --check src/

# Show the diff without modifying
black --diff src/mymodule.py

What Black Changes

Black makes specific, opinionated changes. Here are the most important ones:

1. Trailing commas in multi-line structures

When Black puts a structure on multiple lines, it adds a trailing comma to the last element. This is called the "magic trailing comma" \text{---} if you put a comma after the last item, Black will always keep the structure multi-line. If you remove the trailing comma, Black may collapse it to one line.

# BEFORE \text{---} no trailing comma
result = some_function(
    argument_one,
    argument_two,
    argument_three
)

# AFTER BLACK \text{---} trailing comma added
result = some_function(
    argument_one,
    argument_two,
    argument_three,
)

2. Double quotes normalized

Black converts all string literals to double quotes, with one exception: if a string already contains a double quote, Black leaves it in single quotes to avoid escaping.

# BEFORE
name = 'Alice'
message = "She said 'hello'"
error = "It's broken"

# AFTER BLACK
name = "Alice"
message = "She said 'hello'"    # single \text{---} avoids backslash
error = "It's broken"            # single \text{---} contains apostrophe

3. Long function calls split to multiple lines

# BEFORE \text{---} too long for one line
result = some_very_long_function_name(first_argument, second_argument, third_argument, fourth_argument)

# AFTER BLACK \text{---} split at 88 characters
result = some_very_long_function_name(
    first_argument, second_argument, third_argument, fourth_argument
)

4. Operators at end vs start of continuation lines

# BEFORE
x = (value_one +
     value_two +
     value_three)

# AFTER BLACK \text{---} operators at start of continuation
x = (
    value_one
    + value_two
    + value_three
)

5. Before/after example \text{---} complete function

# BEFORE BLACK
def calculate_discount(price,discount_rate,apply=True,max_discount=100):
    if apply==True:
        discount=price*discount_rate
        if discount>max_discount:
            discount=max_discount
        return price-discount
    else:
        return price

# AFTER BLACK
def calculate_discount(
    price, discount_rate, apply=True, max_discount=100
):
    if apply == True:
        discount = price * discount_rate
        if discount > max_discount:
            discount = max_discount
        return price - discount
    else:
        return price

Note: Black fixes whitespace and structure but does not rename variables, change logic, or enforce PEP 8 rules that flake8 handles (like == True should be if apply:).

Configuring Black

Black intentionally has very few configuration options. The only common ones:

# pyproject.toml
[tool.black]
line-length = 88          # default \text{---} only change if your team has a specific reason
target-version = ["py311"] # minimum Python version to target
include = '\.pyi?$'        # which files to format
extend-exclude = '''
/(
  | migrations
  | generated
)/
'''

The Magic Trailing Comma

This deserves emphasis because it gives you control over Black's splitting behavior:

# WITHOUT trailing comma - Black may collapse to one line if it fits
result = function(arg_one, arg_two, arg_three)

# WITH trailing comma - Black ALWAYS keeps this multi-line
result = function(
    arg_one,
    arg_two,
    arg_three,
)

Use the trailing comma when you want a structure to always be multi-line for readability (e.g., long argument lists, dictionaries with many keys). Remove it to let Black decide based on line length.

isort: Automated Import Sorting

Imports in Python files have a PEP 8 prescribed order: stdlib, then third-party, then local - each group separated by a blank line. Maintaining this order manually across a team is tedious and error-prone. isort does it automatically.

Installation and Usage

pip install isort

# Sort imports in a file
isort src/mymodule.py

# Sort all files in a directory
isort src/

# Check without modifying
isort --check src/

# Show diff
isort --diff src/mymodule.py

Before and After

# BEFORE isort - disordered, wrong grouping, multiple imports on one line
from myapp.models import User
import os, sys
import requests
from django.db import models
import json
from myapp.utils import helpers
from typing import Optional
import datetime

# AFTER isort
import datetime
import json
import os
import sys
from typing import Optional

import requests
from django.db import models

from myapp.models import User
from myapp.utils import helpers

isort automatically:

Groups imports into stdlib / third-party / local
Alphabetizes within each group
Separates groups with blank lines
Splits multi-import lines onto separate lines

Black Compatibility - Critical Configuration

Without proper configuration, isort and Black can conflict. Black formats imports in ways that isort then wants to change, and vice versa. The fix is to use isort's black profile:

# pyproject.toml
[tool.isort]
profile = "black"
line_length = 88
known_first_party = ["myapp", "tests"]

profile = "black" is the most important setting. It configures isort to produce output compatible with Black's formatting, ending the conflict entirely. Always set this.

Configuring Known Packages

isort guesses which packages are stdlib, third-party, and local. It gets this mostly right, but you may need to help it for local packages:

[tool.isort]
profile = "black"
line_length = 88
# Tell isort which names belong to your local application
known_first_party = ["myapp", "api", "models", "utils"]
# Third-party packages that isort might misidentify
known_third_party = ["django", "requests", "pydantic", "fastapi"]

flake8: The Linter That Finds Real Bugs

PEP 8 compliance matters, but finding actual bugs before they reach production matters more. flake8 combines style checking with code analysis to do both.

Installation and Usage

pip install flake8

# Check a single file
flake8 src/mymodule.py

# Check all Python files
flake8 src/

# Show statistics
flake8 --statistics src/

# Show first occurrence only
flake8 --max-line-length 88 src/

Reading flake8 Output

src/payments.py:14:5: F841 local variable 'response' is assigned to but never used
src/payments.py:22:1: E302 expected 2 blank lines, found 1
src/payments.py:28:80: E501 line too long (94 > 88 characters)
src/payments.py:35:19: F821 undefined name 'processs'
src/payments.py:41:9: E711 comparison to None (use "is not" or "is")

Reading the format:

<file>:<line>:<column>: <code> <message>

The Most Valuable flake8 Error Codes

The F-series codes (from pyflakes) are the most valuable because they indicate real bugs:

# F401 - imported but unused (clutters namespace, may indicate copy-paste error)
import os  # never used
from typing import List  # not used anywhere

# F811 - redefinition of unused name (often a copy-paste error)
def validate(data):
    ...

def validate(data, strict=False):  # F811 - first definition is dead
    ...

# F821 - undefined name (will crash at runtime)
def process():
    return processs(data)  # F821 - typo, 'processs' doesn't exist

# F841 - local variable assigned but never used (often a missing return)
def calculate():
    result = expensive_computation()  # F841 - result never returned or used
    return 42

Configuring flake8

Create a .flake8 file in your project root:

[flake8]
max-line-length = 88
# E203: whitespace before ':' - conflicts with Black's slice formatting
# W503: line break before binary operator - conflicts with Black
extend-ignore = E203, W503
exclude =
    .git,
    __pycache__,
    .venv,
    venv,
    build,
    dist,
    *.egg-info,
    migrations
per-file-ignores =
    # F401: __init__.py often re-exports names intentionally
    */__init__.py: F401
    # S101: tests use assert intentionally
    tests/*: S101

flake8 Plugins Worth Installing

# Catches additional code quality issues
pip install flake8-bugbear

# flake8-bugbear catches:
# B006: mutable default arguments (a common Python gotcha)
# B007: loop variable not used in loop body
# B009: getattr/setattr with constant attribute name
# B010: setattr with constant attribute name
# B011: assert False should be raise AssertionError

# B006 - mutable default argument (classic Python bug)
def add_item(item, container=[]):  # B006: default list is shared!
    container.append(item)
    return container

# B007 - loop variable defined but unused
for index in range(10):            # B007: index never used
    print("hello")

# Better:
for _ in range(10):
    print("hello")

mypy: Static Type Checking

Python is dynamically typed - the interpreter checks types at runtime. mypy performs static type analysis, catching type errors before you run the code.

Installation and Usage

pip install mypy

# Check a file
mypy src/mymodule.py

# Check all files
mypy src/

# Strict mode (recommended for new code)
mypy --strict src/

Basic Type Errors mypy Catches

# mypy input
def multiply(a: int, b: int) -> int:
    return a * b

result = multiply("hello", 3)  # Argument 1 to "multiply" has incompatible type "str"; expected "int"

def get_user(user_id: int) -> dict | None:
    if user_id == 1:
        return {"name": "Alice"}
    return None

user = get_user(1)
print(user["name"])  # Item "None" of "dict | None" has no attribute "__getitem__"

mypy caught a potential TypeError and a potential AttributeError that would crash at runtime. These bugs are found statically - before you deploy.

Common mypy Errors and How to Fix Them

Error: "Item ... of ... has no attribute ..."

This happens when you access an attribute on a value that might be None:

# mypy error: Item "None" of "User | None" has no attribute "email"
user = find_user(user_id)
print(user.email)

# FIX 1 - check before use
user = find_user(user_id)
if user is not None:
    print(user.email)

# FIX 2 - assert (when you are certain it is not None)
user = find_user(user_id)
assert user is not None, f"User {user_id} not found"
print(user.email)

# FIX 3 - use walrus operator
if user := find_user(user_id):
    print(user.email)

Error: "Missing return statement"

# mypy error: Missing return statement
def get_status_code(status: str) -> int:
    if status == "ok":
        return 200
    elif status == "error":
        return 500
    # mypy: what about other values of status?

# FIX - add a default or raise
def get_status_code(status: str) -> int:
    if status == "ok":
        return 200
    elif status == "error":
        return 500
    raise ValueError(f"Unknown status: {status!r}")

Error: "Argument ... has incompatible type"

from typing import Sequence

def sum_values(values: list[int]) -> int:
    return sum(values)

# mypy error: Argument 1 has incompatible type "tuple[int, ...]"; expected "list[int]"
result = sum_values((1, 2, 3))

# FIX - use Sequence[int] which accepts both list and tuple
def sum_values(values: Sequence[int]) -> int:
    return sum(values)

Configuring mypy

# pyproject.toml
[tool.mypy]
python_version = "3.11"
strict = true
# Or configure individual strict options:
warn_return_any = true
warn_unused_configs = true
disallow_untyped_defs = true
disallow_any_generics = true
check_untyped_defs = true
disallow_untyped_calls = true

# Per-module overrides (for third-party libraries without type stubs)
[[tool.mypy.overrides]]
module = ["requests.*", "boto3.*", "botocore.*"]
ignore_missing_imports = true

[[tool.mypy.overrides]]
module = "tests.*"
disallow_untyped_defs = false

The `# type: ignore` Escape Hatch

Like # noqa for flake8, # type: ignore tells mypy to skip a line:

# When a third-party library has no type stubs
import some_untyped_library  # type: ignore[import]

# When you are interfacing with dynamic code that mypy cannot analyze
result = getattr(obj, method_name)  # type: ignore[no-any-return]

Always include the specific error code in brackets. Bare # type: ignore suppresses all errors on a line, which is too aggressive. Always add a comment explaining why the suppression is justified.

pre-commit: Hooks That Enforce Quality Locally

pre-commit is a framework for managing git hooks. A git hook is a script that runs automatically at specific points in the git workflow. The pre-commit hook runs before every git commit. If any hook fails, the commit is blocked.

pre-commit hooks catch problems before they enter the repository - before code review, before CI, before merging. This is the fastest possible feedback loop for code quality.

Installation

pip install pre-commit

# Install the hooks into your .git directory
pre-commit install

# Run all hooks against all files (useful after initial setup)
pre-commit run --all-files

After pre-commit install, every git commit will automatically run your hooks.

`.pre-commit-config.yaml` - The Complete Setup

# .pre-commit-config.yaml
repos:
  - repo: https://github.com/pre-commit/pre-commit-hooks
    rev: v4.5.0
    hooks:
      # Prevent committing large files
      - id: check-added-large-files
        args: ["--maxkb=500"]
      # Validate YAML syntax
      - id: check-yaml
      # Validate TOML syntax
      - id: check-toml
      # No trailing whitespace
      - id: trailing-whitespace
      # Files must end with a newline
      - id: end-of-file-fixer
      # Don't commit directly to main/master
      - id: no-commit-to-branch
        args: ["--branch", "main", "--branch", "master"]
      # Catch common merge conflict markers
      - id: check-merge-conflict
      # Validate Python syntax
      - id: check-ast

  - repo: https://github.com/psf/black
    rev: 23.12.1
    hooks:
      - id: black
        language_version: python3.11

  - repo: https://github.com/pycqa/isort
    rev: 5.13.2
    hooks:
      - id: isort
        name: isort (python)

  - repo: https://github.com/pycqa/flake8
    rev: 7.0.0
    hooks:
      - id: flake8
        additional_dependencies:
          - flake8-bugbear
          - flake8-comprehensions

  - repo: https://github.com/pre-commit/mirrors-mypy
    rev: v1.8.0
    hooks:
      - id: mypy
        additional_dependencies:
          - types-requests
          - types-pyyaml
        args: ["--config-file=pyproject.toml"]

How pre-commit Works - Workflow

When you run git commit:

pre-commit intercepts the commit
Runs each hook in the order defined in .pre-commit-config.yaml
If a hook modifies files (like Black and isort do), the commit fails - the files have been changed, but the changes are not staged
You review the changes (they are already correct), stage them with git add, and commit again
If all hooks pass without modifying files, the commit proceeds

$ git commit -m "add user authentication"
[INFO] Initializing environment for https://github.com/psf/black.
[INFO] Installing environment for https://github.com/psf/black.
check yaml...............................................................Passed
trailing whitespace......................................................Passed
end of file fixer........................................................Passed
black....................................................................Failed
- hook id: black
- files were modified by this hook

reformatted src/auth.py

isort....................................................................Passed
flake8...................................................................Passed
mypy.....................................................................Passed

# Black reformatted auth.py. Stage it and commit again:
$ git add src/auth.py
$ git commit -m "add user authentication"
black....................................................................Passed
isort....................................................................Passed
flake8...................................................................Passed
mypy.....................................................................Passed
[main abc1234] add user authentication

Updating Hook Versions

Hook versions are pinned in .pre-commit-config.yaml. To update to the latest versions:

pre-commit autoupdate

This rewrites the rev fields to the latest tagged versions of each repository. Review the changes and commit the updated config file.

Running Hooks Selectively

# Run only one hook
pre-commit run black

# Run all hooks on specific files
pre-commit run --files src/payments.py src/auth.py

# Run all hooks on all files (ignores staged/unstaged distinction)
pre-commit run --all-files

# Skip hooks for one commit (use sparingly)
git commit --no-verify -m "WIP: skip hooks this once"

--no-verify bypasses all hooks. Use it only in genuine emergencies (e.g., you need to commit broken code to unblock a colleague). Never use it to avoid fixing quality issues.

`pyproject.toml` - The Modern Configuration Hub

Python's tool ecosystem historically used different config files: setup.py, setup.cfg, .flake8, mypy.ini, .isort.cfg, etc. pyproject.toml (introduced in PEP 517 and extended by PEP 518) is the modern single-file replacement for most of these.

Complete `pyproject.toml` for a Professional Project

# pyproject.toml

[build-system]
requires = ["setuptools>=68", "wheel"]
build-backend = "setuptools.backends.legacy:build"

[project]
name = "myapp"
version = "1.0.0"
description = "A professional Python application"
readme = "README.md"
requires-python = ">=3.11"
license = {text = "MIT"}
dependencies = [
    "fastapi>=0.104",
    "pydantic>=2.0",
    "sqlalchemy>=2.0",
    "httpx>=0.25",
]

[project.optional-dependencies]
dev = [
    "black>=23.0",
    "isort>=5.12",
    "flake8>=7.0",
    "flake8-bugbear>=23.0",
    "mypy>=1.5",
    "pre-commit>=3.5",
    "pytest>=7.4",
    "pytest-asyncio>=0.21",
]

# ─── Black configuration ──────────────────────────────────────────────────────
[tool.black]
line-length = 88
target-version = ["py311"]
extend-exclude = '''
/(
  | migrations
  | generated_code
)/
'''

# ─── isort configuration ──────────────────────────────────────────────────────
[tool.isort]
profile = "black"
line_length = 88
known_first_party = ["myapp"]
known_third_party = ["fastapi", "pydantic", "sqlalchemy"]

# ─── mypy configuration ───────────────────────────────────────────────────────
[tool.mypy]
python_version = "3.11"
strict = true
pretty = true

[[tool.mypy.overrides]]
module = "tests.*"
disallow_untyped_defs = false

[[tool.mypy.overrides]]
module = ["boto3.*", "botocore.*"]
ignore_missing_imports = true

# ─── pytest configuration ─────────────────────────────────────────────────────
[tool.pytest.ini_options]
testpaths = ["tests"]
asyncio_mode = "auto"
addopts = "-v --tb=short"

# Note: flake8 does not support pyproject.toml natively.
# Use a .flake8 file alongside this for flake8 configuration.

The flake8 / pyproject.toml Exception

As of 2024, flake8 does not natively read pyproject.toml. This is a known limitation of flake8's architecture. The solutions are:

Keep a .flake8 file alongside pyproject.toml (most common approach)
Install flake8-pyproject plugin which adds pyproject.toml support to flake8
Switch to ruff which reads pyproject.toml natively and is orders of magnitude faster

# Option 3: ruff (increasingly popular replacement for flake8 + isort)
pip install ruff

# pyproject.toml ruff config
# [tool.ruff]
# line-length = 88
# select = ["E", "F", "W", "I"]  # E/W: pycodestyle, F: pyflakes, I: isort

GitHub Actions CI Pipeline

Local pre-commit hooks catch most problems. CI catches the rest - changes made without hooks (e.g., direct pushes, edits via GitHub's web interface, or team members who have not installed pre-commit).

Complete GitHub Actions Workflow

# .github/workflows/quality.yml
name: Code Quality

on:
  push:
    branches: [main, develop]
  pull_request:
    branches: [main, develop]

jobs:
  quality:
    name: Lint, Format, Type Check
    runs-on: ubuntu-latest

    steps:
      - name: Checkout code
        uses: actions/checkout@v4

      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: "3.11"
          cache: "pip"

      - name: Install dependencies
        run: |
          python -m pip install --upgrade pip
          pip install -e ".[dev]"

      - name: Run Black (format check)
        run: black --check --diff src/ tests/

      - name: Run isort (import order check)
        run: isort --check --diff src/ tests/

      - name: Run flake8 (linting)
        run: flake8 src/ tests/

      - name: Run mypy (type checking)
        run: mypy src/

  test:
    name: Tests
    runs-on: ubuntu-latest
    needs: quality  # only run tests if quality checks pass

    steps:
      - uses: actions/checkout@v4

      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: "3.11"
          cache: "pip"

      - name: Install dependencies
        run: |
          python -m pip install --upgrade pip
          pip install -e ".[dev]"

      - name: Run tests
        run: pytest tests/ --tb=short -q

The CI Pipeline Mindset

The relationship between local hooks and CI:

Local hooks: fast feedback (milliseconds to seconds), runs only on changed files, developer sees the issue immediately

CI pipeline: authoritative gate, runs on the full codebase, catches anything that slipped past local hooks (teammate who skipped pre-commit install, hotfix committed without hooks, etc.)

The key principle: CI should never reject something that would pass local checks. The tools are the same, the configuration is the same. If CI fails for a reason that does not fail locally, your config is inconsistent.

Building the Toolchain From Scratch

Let's put everything together. Here is the complete setup sequence for a new Python project:

# 1. Create the project
mkdir myproject && cd myproject
python -m venv .venv
source .venv/bin/activate  # Windows: .venv\Scripts\activate

# 2. Install all quality tools
pip install black isort flake8 flake8-bugbear mypy pre-commit

# 3. Create pyproject.toml
cat > pyproject.toml << 'EOF'
[build-system]
requires = ["setuptools>=68"]
build-backend = "setuptools.backends.legacy:build"

[project]
name = "myproject"
version = "0.1.0"
requires-python = ">=3.11"

[project.optional-dependencies]
dev = ["black", "isort", "flake8", "flake8-bugbear", "mypy", "pre-commit"]

[tool.black]
line-length = 88
target-version = ["py311"]

[tool.isort]
profile = "black"
line_length = 88

[tool.mypy]
python_version = "3.11"
strict = true
EOF

# 4. Create .flake8
cat > .flake8 << 'EOF'
[flake8]
max-line-length = 88
extend-ignore = E203, W503
exclude = .git, __pycache__, .venv, build, dist
per-file-ignores =
    */__init__.py: F401
EOF

# 5. Create .pre-commit-config.yaml
cat > .pre-commit-config.yaml << 'EOF'
repos:
  - repo: https://github.com/pre-commit/pre-commit-hooks
    rev: v4.5.0
    hooks:
      - id: trailing-whitespace
      - id: end-of-file-fixer
      - id: check-yaml
      - id: check-toml
      - id: check-ast
      - id: check-merge-conflict

  - repo: https://github.com/psf/black
    rev: 23.12.1
    hooks:
      - id: black

  - repo: https://github.com/pycqa/isort
    rev: 5.13.2
    hooks:
      - id: isort

  - repo: https://github.com/pycqa/flake8
    rev: 7.0.0
    hooks:
      - id: flake8
        additional_dependencies: [flake8-bugbear]

  - repo: https://github.com/pre-commit/mirrors-mypy
    rev: v1.8.0
    hooks:
      - id: mypy
        args: ["--config-file=pyproject.toml"]
EOF

# 6. Initialize git and install hooks
git init
pre-commit install

# 7. Run all hooks on initial codebase
pre-commit run --all-files

# 8. Verify everything is clean
echo "Toolchain configured. Pre-commit hooks installed."
git add .
git commit -m "chore: configure code quality toolchain"

Ruff - The Next Generation (Bonus)

While this lesson covers the classic toolchain (Black + isort + flake8), it is worth mentioning Ruff: a modern Python linter and formatter written in Rust. Ruff is 10-100x faster than flake8, replaces isort, and has recently added a formatter to compete with Black.

pip install ruff

# Run linting
ruff check src/

# Auto-fix issues
ruff check --fix src/

# Format (Black-compatible)
ruff format src/

# pyproject.toml ruff configuration
[tool.ruff]
line-length = 88
target-version = "py311"

[tool.ruff.lint]
select = [
    "E",    # pycodestyle errors
    "W",    # pycodestyle warnings
    "F",    # pyflakes
    "I",    # isort
    "B",    # flake8-bugbear
    "C4",   # flake8-comprehensions
    "UP",   # pyupgrade (modernize old Python patterns)
]
ignore = ["E203", "W503"]

[tool.ruff.lint.isort]
known-first-party = ["myapp"]

Ruff's advantage is speed - on a large codebase, flake8 might take 30 seconds; Ruff takes under a second. For new projects, Ruff is increasingly the recommended choice. For existing projects already using Black + flake8, the migration is straightforward.

Interview Questions

Q1: What is Black and why is its "opinionated" design intentional?

Answer: Black is a Python code formatter released in 2018 that reformats Python source code to its own style automatically. Its defining characteristic is that it is deliberately opinionated and non-configurable on most style choices. This is intentional: the primary value of a code formatter is not that it produces beautiful code, but that it ends formatting debates. When there are no style choices to make (because Black has already made them), teams stop spending time in code review discussing formatting. The goal is to move all formatting decisions from humans to the tool, permanently. Black achieves this by being so opinionated that the only question a team needs to answer is "do we use Black?" rather than a hundred individual style questions.

Q2: Why must isort be configured with `profile = "black"` when used alongside Black?

Answer: Black and isort can produce conflicting formatting for import statements. Black has specific opinions about how multi-line imports should be formatted (e.g., how continuation lines are indented, how trailing commas are placed). Without configuration, isort may reformat imports to pass its own validation, then Black reformats them again to pass its validation, creating an infinite loop where neither tool is satisfied. Setting profile = "black" in isort's configuration tells isort to produce output that matches Black's expected style, so both tools agree on the final format. Without this setting, engineers running both tools locally will encounter files that perpetually ping-pong between two formats.

Q3: What is the difference between a linter and a formatter?

Answer: A linter analyzes code and reports problems without modifying anything. A formatter analyzes code and rewrites it to match a style, producing a modified file as output. flake8 is a linter - it tells you "line 14 has a trailing whitespace" and stops. Black is a formatter - it removes the trailing whitespace and saves the file. In practice, a complete quality toolchain needs both: formatters to handle all the mechanical style issues automatically, and linters to catch actual bugs and patterns that formatters do not address (undefined names, unused imports, mutable default arguments, overly complex functions). The distinction matters operationally: formatters should run first (and their output is then checked by linters), and in CI, formatters run in --check mode (they report problems but do not modify files).

Q4: What are pre-commit hooks and why are they valuable over relying solely on CI?

Answer: Pre-commit hooks are scripts configured to run automatically before every git commit. They run locally on the developer's machine, before code is pushed to a remote repository. Their value over CI-only enforcement is speed of feedback: a pre-commit hook runs in seconds and shows the engineer the problem immediately, before the code ever leaves their machine. CI feedback has a latency of minutes (waiting for the pipeline to start, clone, install, run checks). Faster feedback is cheaper - a problem caught locally in 3 seconds costs far less to fix than one caught 10 minutes later in CI, when the engineer has switched context. Pre-commit and CI complement each other: hooks provide the fast local feedback loop, CI provides the authoritative gate.

Q5: What does `mypy --strict` enforce that the default mode does not?

Answer: By default, mypy only checks functions and variables that have type annotations - it ignores unannotated code. --strict mode enforces several additional requirements: all functions must have type annotations on all parameters and return values (disallow_untyped_defs); generic types cannot be used without parameters like list instead of list[str] (disallow_any_generics); functions returning Any type are flagged (warn_return_any); calling untyped functions is flagged (disallow_untyped_calls). In strict mode, mypy acts as an enforcer: if you write Python code without type annotations, mypy considers it an error. This is the recommended setting for new code because it ensures the type system provides maximum coverage. For legacy code without annotations, --strict is introduced incrementally per-module.

Q6: What does the magic trailing comma do in Black, and how do you use it intentionally?

Answer: The magic trailing comma is a trailing comma after the last element in a multi-line collection or function call. Black treats a trailing comma as a signal to always keep the structure multi-line, regardless of whether it would fit on one line. Without a trailing comma, Black may collapse a multi-line structure to a single line if it fits within the line length limit. With a trailing comma, Black always expands it. Engineers use this intentionally in two ways: add a trailing comma to structures that you want to always be multi-line for readability (long function signatures, configuration dictionaries, lists of many items); remove the trailing comma to let Black decide based on line length. This gives engineers precise control over layout through a single character, without fighting the formatter.

Practice Challenges

Beginner - Set Up the Full Toolchain

Set up a complete toolchain for a new project. The project has a single file app.py with the following content. Configure Black, isort, flake8, and pre-commit, then run them all.

import sys,os
import requests
from typing import List,Optional
import json

API_URL="https://api.example.com"
TIMEOUT=30

def fetchUsers(api_key:str,limit:int=10)->List[dict]:
    headers={'Authorization':f'Bearer {api_key}','Content-Type':'application/json'}
    resp=requests.get(f"{API_URL}/users",headers=headers,params={'limit':limit},timeout=TIMEOUT)
    if resp.status_code==200:
        return resp.json()['users']
    else:
        return []

def processUser(u:dict)->Optional[dict]:
    if u==None:
        return None
    return {'id':u['id'],'name':u['name'],'email':u['email'].lower()}

def main():
    key=os.environ.get('API_KEY','')
    if not key:
        print("No API key")
        sys.exit(1)
    users=fetchUsers(key)
    results=[]
    for u in users:
        r=processUser(u)
        if r!=None:
            results.append(r)
    print(json.dumps(results,indent=2))

if __name__=='__main__':
    main()

Solution

Step 1: Create configuration files

# pyproject.toml
[tool.black]
line-length = 88
target-version = ["py311"]

[tool.isort]
profile = "black"
line_length = 88

[tool.mypy]
python_version = "3.11"
disallow_untyped_defs = true

# .flake8
[flake8]
max-line-length = 88
extend-ignore = E203, W503

# .pre-commit-config.yaml
repos:
  - repo: https://github.com/pre-commit/pre-commit-hooks
    rev: v4.5.0
    hooks:
      - id: trailing-whitespace
      - id: end-of-file-fixer
      - id: check-ast

  - repo: https://github.com/psf/black
    rev: 23.12.1
    hooks:
      - id: black

  - repo: https://github.com/pycqa/isort
    rev: 5.13.2
    hooks:
      - id: isort

  - repo: https://github.com/pycqa/flake8
    rev: 7.0.0
    hooks:
      - id: flake8

Step 2: Run tools

# Install and set up
pip install black isort flake8 pre-commit
pre-commit install

# Format
black app.py
isort app.py

# After formatting, the file looks like:

Resulting app.py after Black + isort:

import json
import os
import sys
from typing import Optional

import requests

API_URL = "https://api.example.com"
TIMEOUT = 30


def fetch_users(api_key: str, limit: int = 10) -> list[dict]:
    headers = {
        "Authorization": f"Bearer {api_key}",
        "Content-Type": "application/json",
    }
    resp = requests.get(
        f"{API_URL}/users",
        headers=headers,
        params={"limit": limit},
        timeout=TIMEOUT,
    )
    if resp.status_code == 200:
        return resp.json()["users"]
    else:
        return []


def process_user(user: dict) -> Optional[dict]:
    if user is None:
        return None
    return {
        "id": user["id"],
        "name": user["name"],
        "email": user["email"].lower(),
    }


def main() -> None:
    key = os.environ.get("API_KEY", "")
    if not key:
        print("No API key")
        sys.exit(1)
    users = fetch_users(key)
    results = []
    for user in users:
        processed = process_user(user)
        if processed is not None:
            results.append(processed)
    print(json.dumps(results, indent=2))


if __name__ == "__main__":
    main()

Issues fixed beyond formatting:

fetchUsers and processUser renamed to snake_case
Single-letter variables u and r renamed to user and processed
u==None changed to u is None, r!=None to processed is not None
List[dict] updated to list[dict] (Python 3.9+ style)
Return type added to main()

Intermediate - Fix a CI Pipeline

The following GitHub Actions workflow has 5 bugs or misconfigurations that would cause it to fail or be inconsistent. Identify and fix each one.

name: Quality Checks

on:
  push:
    branches: [main]

jobs:
  lint:
    runs-on: ubuntu-20.04
    steps:
      - uses: actions/checkout@v3

      - name: Set up Python
        uses: actions/setup-python@v4
        with:
          python-version: "3.9"

      - name: Install dependencies
        run: pip install black isort flake8

      - name: Black check
        run: black src/

      - name: isort check
        run: isort src/

      - name: flake8
        run: flake8 --max-line-length=79 src/

Solution

Bug 1: black src/ modifies files in CI instead of checking

In CI, formatters should run in check mode:

- name: Black check
  run: black --check --diff src/

Bug 2: isort src/ modifies files in CI instead of checking

- name: isort check
  run: isort --check --diff src/

Bug 3: --max-line-length=79 conflicts with Black's 88-character default

Black formats to 88 characters. If flake8 checks at 79, it will flag lines that Black just produced as too long. They must match:

- name: flake8
  run: flake8 --max-line-length=88 --extend-ignore=E203,W503 src/

Bug 4: Python version mismatch with local development

Using Python 3.9 in CI but modern features (like list[int] without from __future__ import annotations) may require 3.10+. Ensure CI matches your pyproject.toml python-version:

python-version: "3.11"  # match pyproject.toml target-version

Bug 5: Missing pull_request trigger - only runs on push to main

Quality checks should run on pull requests, not just after merge:

on:
  push:
    branches: [main]
  pull_request:
    branches: [main]

Fixed workflow:

name: Quality Checks

on:
  push:
    branches: [main]
  pull_request:
    branches: [main]

jobs:
  lint:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: "3.11"
          cache: "pip"

      - name: Install dependencies
        run: pip install black isort flake8

      - name: Black (format check)
        run: black --check --diff src/

      - name: isort (import order check)
        run: isort --check --diff src/

      - name: flake8 (linting)
        run: flake8 --max-line-length=88 --extend-ignore=E203,W503 src/

Advanced - Build a Custom mypy Plugin

Write a mypy plugin that enforces a custom rule: any function decorated with @api_endpoint must have a return type annotation. Functions without this decorator are not checked.

Solution

# mypy_api_endpoint_plugin.py
"""
mypy plugin that requires return type annotations on @api_endpoint functions.

Install by adding to pyproject.toml:
    [tool.mypy]
    plugins = ["mypy_api_endpoint_plugin"]
"""

from mypy.plugin import Plugin, FunctionContext
from mypy.types import Type, NoneType
from mypy.nodes import Decorator, FuncDef
from typing import Callable, Optional


class ApiEndpointPlugin(Plugin):
    """
    mypy plugin that enforces return type annotations
    on functions decorated with @api_endpoint.
    """

    def get_function_hook(
        self, fullname: str
    ) -> Optional[Callable[[FunctionContext], Type]]:
        """Called for every function call mypy encounters."""
        return None  # We use get_method_hook instead for decorators

    def get_decorator_hook(
        self, fullname: str
    ) -> Optional[Callable]:
        """Called when mypy processes a decorator."""
        if fullname.endswith("api_endpoint"):
            return self._check_api_endpoint
        return None

    def _check_api_endpoint(self, ctx) -> None:
        """Check that a function decorated with @api_endpoint has a return type."""
        # Access the decorated function
        if isinstance(ctx.reason, Decorator):
            func = ctx.reason.func
            if isinstance(func, FuncDef):
                if func.type is None or not hasattr(func.type, "ret_type"):
                    ctx.api.fail(
                        f'Function "{func.name}" is decorated with @api_endpoint '
                        f"but has no return type annotation. "
                        f"API endpoints must declare their return type explicitly.",
                        ctx.reason,
                    )


def plugin(version: str):
    """Entry point for mypy plugin discovery."""
    return ApiEndpointPlugin


# ─── Example usage ────────────────────────────────────────────────────────────

# In a real project, api_endpoint would be defined somewhere:
def api_endpoint(func):
    """Decorator that marks a function as an API endpoint."""
    return func


# This should FAIL mypy with the plugin:
@api_endpoint
def get_users(page: int):  # no return type
    return []


# This should PASS:
@api_endpoint
def get_user(user_id: int) -> dict:
    return {"id": user_id}


# This function without the decorator should be ignored:
def helper(x):  # no decorator - plugin ignores it
    return x * 2

# pyproject.toml - enable the plugin
[tool.mypy]
python_version = "3.11"
plugins = ["mypy_api_endpoint_plugin"]

# Test the plugin
mypy --config-file pyproject.toml example.py
# example.py:XX: error: Function "get_users" is decorated with @api_endpoint
# but has no return type annotation. API endpoints must declare their return type explicitly.

This demonstrates how to extend mypy's type system with custom rules that enforce project-specific conventions. The same pattern can enforce naming conventions, required docstrings, or compliance with internal APIs.

Quick Reference

Tool	Purpose	Check mode	Fix mode
`black`	Auto-formatter	`black --check src/`	`black src/`
`isort`	Import sorter	`isort --check src/`	`isort src/`
`flake8`	Linter (style + bugs)	`flake8 src/`	N/A (reports only)
`mypy`	Static type checker	`mypy src/`	N/A (reports only)
`pre-commit`	Hook manager	`pre-commit run --all-files`	(hooks fix automatically)

Config file	What it configures
`pyproject.toml`	Black, isort, mypy, pytest
`.flake8`	flake8 (no native pyproject.toml support)
`.pre-commit-config.yaml`	pre-commit hook definitions
`.github/workflows/*.yml`	GitHub Actions CI pipeline

pre-commit command	Effect
`pre-commit install`	Install hooks into `.git/hooks/`
`pre-commit run --all-files`	Run all hooks on entire codebase
`pre-commit run black`	Run only the black hook
`pre-commit autoupdate`	Update all hooks to latest versions
`git commit --no-verify`	Bypass hooks (use sparingly)

Key Takeaways

Automated formatters end style debates permanently. Black reformats code to its style without negotiation, and the correct response to "should we use Black?" is almost always yes.
isort and Black must be configured to agree: profile = "black" in isort's config is the single most important isort setting for any project that also uses Black.
flake8's F-series error codes (undefined names, unused imports) detect real bugs, not just style issues - these are higher priority than E/W codes in code review.
mypy --strict mode maximizes type safety by requiring annotations everywhere and disallowing Any types. Apply it to new code; migrate legacy code incrementally per module.
pre-commit hooks provide the fastest possible feedback on quality issues - before code is pushed, before CI starts, before anyone else sees the problem.
The magic trailing comma in Black gives engineers control over line splitting: a trailing comma forces multi-line layout; removing it lets Black decide based on line length.
A consistent toolchain (same tools, same config, in both pre-commit hooks and CI) ensures that what passes locally always passes in CI, eliminating the "but it worked on my machine" class of CI failures.

What You Will Learn​

Prerequisites​

The End of the Tabs vs Spaces Wars​

Black: The Opinionated Formatter​

Installation​

Basic Usage​

What Black Changes​

Configuring Black​

The Magic Trailing Comma​

isort: Automated Import Sorting​

Installation and Usage​

Before and After​

Black Compatibility - Critical Configuration​

Configuring Known Packages​

flake8: The Linter That Finds Real Bugs​

Installation and Usage​

Reading flake8 Output​

The Most Valuable flake8 Error Codes​

Configuring flake8​

flake8 Plugins Worth Installing​

mypy: Static Type Checking​

Installation and Usage​

Basic Type Errors mypy Catches​

Common mypy Errors and How to Fix Them​

Configuring mypy​

The # type: ignore Escape Hatch​

pre-commit: Hooks That Enforce Quality Locally​

Installation​

.pre-commit-config.yaml - The Complete Setup​

How pre-commit Works - Workflow​

Updating Hook Versions​

Running Hooks Selectively​

pyproject.toml - The Modern Configuration Hub​

Complete pyproject.toml for a Professional Project​

The flake8 / pyproject.toml Exception​

GitHub Actions CI Pipeline​

Complete GitHub Actions Workflow​

The CI Pipeline Mindset​

Building the Toolchain From Scratch​

Ruff - The Next Generation (Bonus)​

Interview Questions​

Q1: What is Black and why is its "opinionated" design intentional?​

Q2: Why must isort be configured with profile = "black" when used alongside Black?​

Q3: What is the difference between a linter and a formatter?​

Q4: What are pre-commit hooks and why are they valuable over relying solely on CI?​

Q5: What does mypy --strict enforce that the default mode does not?​

Q6: What does the magic trailing comma do in Black, and how do you use it intentionally?​

Practice Challenges​

Beginner - Set Up the Full Toolchain​

Intermediate - Fix a CI Pipeline​

Advanced - Build a Custom mypy Plugin​

Quick Reference​

Key Takeaways​

What You Will Learn

Prerequisites

The End of the Tabs vs Spaces Wars

Black: The Opinionated Formatter

Installation

Basic Usage

What Black Changes

Configuring Black

The Magic Trailing Comma

isort: Automated Import Sorting

Installation and Usage

Before and After

Black Compatibility - Critical Configuration

Configuring Known Packages

flake8: The Linter That Finds Real Bugs

Installation and Usage

Reading flake8 Output

The Most Valuable flake8 Error Codes

Configuring flake8

flake8 Plugins Worth Installing

mypy: Static Type Checking

Installation and Usage

Basic Type Errors mypy Catches

Common mypy Errors and How to Fix Them

Configuring mypy

The `# type: ignore` Escape Hatch

pre-commit: Hooks That Enforce Quality Locally

Installation

`.pre-commit-config.yaml` - The Complete Setup

How pre-commit Works - Workflow

Updating Hook Versions

Running Hooks Selectively

`pyproject.toml` - The Modern Configuration Hub

Complete `pyproject.toml` for a Professional Project

The flake8 / pyproject.toml Exception

GitHub Actions CI Pipeline

Complete GitHub Actions Workflow

The CI Pipeline Mindset

Building the Toolchain From Scratch

Ruff - The Next Generation (Bonus)

Interview Questions

Q1: What is Black and why is its "opinionated" design intentional?

Q2: Why must isort be configured with `profile = "black"` when used alongside Black?

Q3: What is the difference between a linter and a formatter?

Q4: What are pre-commit hooks and why are they valuable over relying solely on CI?

Q5: What does `mypy --strict` enforce that the default mode does not?

Q6: What does the magic trailing comma do in Black, and how do you use it intentionally?

Practice Challenges

Beginner - Set Up the Full Toolchain

Intermediate - Fix a CI Pipeline

Advanced - Build a Custom mypy Plugin

Quick Reference

Key Takeaways