Linting and Formatting - Ruff, Black, isort, and mypy

Reading time: ~30 minutes | Level: Intermediate → Engineering

Before reading further, count the style and lint violations in this code:

import os,sys
import json

def process( data,output_file ):
    x=json.dumps(data)
    f=open(output_file,'w')
    f.write(x)
    return None

Show Answer

There are at least 8 violations:

1. import os,sys - two imports on one line (E401)
2. import os,sys - os and sys are imported but never used (F401 × 2)
3. import json - imported after other imports but json is used; import order is wrong relative to stdlib grouping (I001)
4. def process( data,output_file ): - space after ( and before ) (E201, E202)
5. def process( data,output_file ): - missing space after comma (E231)
6. x=json.dumps(data) - missing spaces around = (E225)
7. f=open(output_file,'w') - open() without context manager - file may not be closed on exception (B019 / UP015 / consider SIM115)
8. return None - explicit return None is redundant; return or nothing suffices (RET504 / implicit)

The code runs correctly on the happy path. That is the point: linting catches violations that do not cause runtime failures but do cause slower reviews, subtle bugs under error conditions, and maintainability problems. The unclosed file handle on exception is a real bug hiding behind "it works."

The corrected version:

import json


def process(data: dict, output_file: str) -> None:
    serialized = json.dumps(data)
    with open(output_file, "w") as f:
        f.write(serialized)

Every violation above is automatable - a linter catches it in under 100 milliseconds. This lesson covers the tools that make that happen.

What You Will Learn

• The conceptual difference between linting and formatting, and why both matter
• Ruff: the modern unified linter written in Rust - rule categories, auto-fix, configuration
• Black: the opinionated formatter - what it changes and why the "no configuration" philosophy works
• isort: import sorting and Black compatibility
• mypy: static type checking - annotations, common errors, strict mode
• How to wire all tools together in pyproject.toml
• A complete developer workflow: edit → lint → format → type-check → commit

Prerequisites

• Lesson 02 (pytest) - understanding test workflows that linting integrates with
• Lesson 05 (Code Coverage) - tooling context; linting is another quality gate
• Basic familiarity with pyproject.toml as a project configuration file

Part 1 - Linting vs Formatting: The Distinction

These terms are often conflated but they do different things:

Tool Type	What it does	Examples
Linter	Reads code, identifies violations, reports them. May auto-fix simple cases.	Ruff, flake8, pylint, mypy
Formatter	Rewrites code to a canonical style. Opinionated. No report, just output.	Black, autopep8, yapf
Type Checker	Statically analyzes type annotations to find type errors before runtime.	mypy, pyright, pytype

The practical distinction: a formatter changes your file deterministically - run it twice and you get the same result. A linter reports problems - you decide how to fix them (though modern linters like Ruff can fix many automatically).

Why consistent style matters beyond aesthetics:

1. Code review noise: If formatting is not enforced, diffs include cosmetic changes mixed with logic changes. Reviewers must mentally separate them.
2. Real bugs: Many lint rules catch actual bugs - unused imports shadowing real ones, mutable default arguments, open files without context managers.
3. Onboarding speed: Consistent style means a new engineer reads any file in the codebase and finds the same patterns.
4. Merge conflicts: Automatic formatting reduces the surface area of conflicts caused by different developers' editors.

Part 2 - Ruff: The Modern All-in-One Linter

Ruff is a Python linter written in Rust. It is 10–100x faster than flake8, and in a single tool it replaces flake8, isort, pyupgrade, pydocstyle, flake8-bugbear, and dozens of flake8 plugins.

# Install
pip install ruff

# Check the current directory
ruff check .

# Check and auto-fix safe violations
ruff check --fix .

# Format (Black-compatible formatter)
ruff format .

# Check a specific file
ruff check src/mymodule.py

Rule Categories

Ruff organizes rules into prefixed categories. You enable them by prefix:

Prefix	Source	What it checks
E	pycodestyle	Style: spacing, line length, blank lines
W	pycodestyle	Warnings: whitespace, deprecated syntax
F	Pyflakes	Logical: unused imports, undefined names, redefined variables
I	isort	Import order and grouping
N	pep8-naming	Naming conventions: classes, functions, constants
UP	pyupgrade	Modern Python syntax: f"{x}" over "{}".format(x), list[int] over List[int]
B	flake8-bugbear	Likely bugs: mutable defaults, loop variable capture, bare except
S	flake8-bandit	Security: exec, subprocess, SQL injection patterns
ANN	flake8-annotations	Missing type annotations
D	pydocstyle	Docstring conventions
RET	flake8-return	Return statement issues
SIM	flake8-simplify	Simplifiable code patterns
C90	mccabe	Cyclomatic complexity

pyproject.toml Configuration

[tool.ruff]
# Target Python version - affects which UP rules apply
target-version = "py311"

# Maximum line length (Black default is 88)
line-length = 88

[tool.ruff.lint]
# Enable specific rule categories
select = [
    "E",    # pycodestyle errors
    "W",    # pycodestyle warnings
    "F",    # Pyflakes
    "I",    # isort
    "N",    # pep8-naming
    "UP",   # pyupgrade
    "B",    # flake8-bugbear
    "S",    # flake8-bandit (security)
    "RET",  # flake8-return
    "SIM",  # flake8-simplify
]

# Ignore specific rules
ignore = [
    "E501",   # line too long - let Black handle line length
    "S101",   # use of assert - fine in test files
    "B008",   # do not perform function calls in default arguments - sometimes intentional
]

# Auto-fix these rule categories (safe to apply automatically)
fixable = ["E", "W", "F", "I", "UP", "RET", "SIM"]

[tool.ruff.lint.per-file-ignores]
# Tests can use assert and don't need all security checks
"tests/**/*.py" = ["S", "ANN"]
# Scripts may import things at odd locations
"scripts/**/*.py" = ["E402"]

[tool.ruff.format]
# Use double quotes (matches Black)
quote-style = "double"
# Indent with spaces
indent-style = "space"

Inline Suppression: # noqa

To suppress a specific rule on a single line:

import os  # noqa: F401  - imported for side effects
SECRET_KEY = "hardcoded-for-dev"  # noqa: S105

danger

# noqa comments silence linting for that line. Overuse defeats the purpose of having a linter. Each # noqa should have a specific rule code (not bare # noqa) and ideally a comment explaining why it is necessary. Treat a bare # noqa the same as a TODO that needs follow-up - it is technical debt.

Part 3 - Black: The Opinionated Formatter

Black is the dominant Python formatter. Its defining feature is its philosophy: no configuration. You cannot tell Black to use single quotes, or to put trailing commas differently, or to change its line length (beyond one setting). This removes all style debates from code review permanently.

# Format all Python files
black .

# Check without modifying (exit code 1 if any file would change)
black --check .

# Show a diff of what Black would change
black --diff .

# Format a specific file
black src/mymodule.py

What Black Changes

Black transforms code to its canonical form. Key changes:

1. Quotes - always double

# Before
x = 'hello'
y = "world"

# After
x = "hello"
y = "world"

2. Trailing commas in multi-line collections

# Before
result = some_function(
    argument_one,
    argument_two,
    argument_three
)

# After - Black adds trailing comma, which means adding a new argument never touches existing lines
result = some_function(
    argument_one,
    argument_two,
    argument_three,
)

3. Line wrapping - magic trailing comma

If a collection fits on one line, Black collapses it. If it does not fit (over 88 characters), Black expands it. The magic trailing comma forces expansion even if it fits:

# No trailing comma - Black may collapse to one line if it fits
result = [1, 2, 3]

# Magic trailing comma - Black always expands this, regardless of length
result = [
    1,
    2,
    3,
]

4. Blank lines - two between top-level definitions, one between methods

# Before
class Foo:
    def method_a(self): pass


    def method_b(self): pass
def top_level(): pass

# After
class Foo:
    def method_a(self):
        pass

    def method_b(self):
        pass


def top_level():
    pass

pyproject.toml Configuration (Minimal by Design)

[tool.black]
# The only setting most teams change
line-length = 88

# Target Python versions (affects syntax choices)
target-version = ["py310", "py311", "py312"]

# Files to exclude (beyond .gitignore)
extend-exclude = '''
/(
  | migrations
  | .venv
)/
'''

warning

Formatter debates are over. Black won. Configure your editor to run Black on save - every editor supports this (VS Code: "editor.formatOnSave": true with the Black extension; PyCharm: External Tools → Black). When Black formats on save, you never think about formatting again. The only remaining debate - tabs vs spaces, quote style, trailing commas - is settled by Black automatically.

Part 4 - isort: Import Sorting

isort sorts and groups Python imports into three sections separated by blank lines:

1. Standard library imports (import os, import sys)
2. Third-party imports (import requests, import django)
3. Local/first-party imports (from myapp import models)

pip install isort

# Sort imports in all files
isort .

# Check without modifying
isort --check-only .

# Show diff
isort --diff .

Why import order matters:

• Consistent ordering reduces diff noise - adding a new import always goes to the same place
• Separating stdlib/third-party/local makes dependencies visually obvious
• Misplaced imports are a common source of merge conflicts

Configuring isort for Black Compatibility

isort and Black have a subtle conflict: isort may format multi-line imports in a way Black immediately reformats differently. The black profile resolves this:

[tool.isort]
# Must match Black's line length
line_length = 88

# The black profile sets multi_line_output, force_grid_wrap, etc.
# to values compatible with Black's output
profile = "black"

# Known first-party packages (your project's packages)
known_first_party = ["myapp", "mylib"]

tip

Use Ruff as your primary linter - it includes isort-compatible import sorting built in (the I rule category). Enable "I" in your Ruff select list and you get import sorting without installing isort separately. Ruff replaces flake8, isort, pyupgrade, and many flake8 plugins in one tool with one configuration. Only add standalone isort if you have a specific workflow that requires it independently.

Part 5 - mypy: Static Type Checking

mypy is a static type checker. It reads your type annotations and proves - at analysis time, without running the code - that your types are consistent.

pip install mypy

# Check all Python files in src/
mypy src/

# Check a specific file
mypy src/mymodule.py

# Strict mode - enables all optional checks
mypy --strict src/

# Ignore imports that have no type stubs
mypy --ignore-missing-imports src/

Type Annotations Recap

from typing import Optional, Union

# Basic types
def greet(name: str) -> str:
    return f"Hello, {name}"

# Optional - value or None
def find_user(user_id: int) -> Optional[str]:
    ...

# Union - one of several types
def process(value: Union[int, str]) -> str:
    return str(value)

# Python 3.10+ union syntax (preferred)
def process_modern(value: int | str) -> str:
    return str(value)

# Collections - generic syntax (Python 3.9+)
def top_items(items: list[int], n: int) -> list[int]:
    return sorted(items, reverse=True)[:n]

def lookup(mapping: dict[str, int], key: str) -> int | None:
    return mapping.get(key)

# Callable
from collections.abc import Callable

def apply(fn: Callable[[int], str], value: int) -> str:
    return fn(value)

Common mypy Errors and Fixes

Error: Argument 1 to "X" has incompatible type "None"; expected "str"

# mypy error
def greet(name: str) -> str:
    return f"Hello, {name}"

user: str | None = get_user()
greet(user)  # error: Argument 1 has incompatible type "str | None"; expected "str"

# Fix: check for None first
if user is not None:
    greet(user)  # mypy now knows user is str

# Or use assert
assert user is not None
greet(user)

Error: Return type "None" incompatible with return type "str"

# mypy error
def find(items: list[str], target: str) -> str:
    for item in items:
        if item == target:
            return item
    # implicit return None - but return type says str

# Fix: return str | None, or raise
def find(items: list[str], target: str) -> str | None:
    for item in items:
        if item == target:
            return item
    return None

Error: Item "None" of "X | None" has no attribute "Y"

result: str | None = maybe_string()
print(result.upper())  # error: Item "None" of "str | None" has no attribute "upper"

# Fix: narrow the type
if result is not None:
    print(result.upper())

pyproject.toml mypy Configuration

[tool.mypy]
# Python version to check against
python_version = "3.11"

# Paths to type-check
files = ["src", "tests"]

# Start with these - less aggressive than --strict
warn_return_any = true
warn_unused_configs = true
warn_unused_ignores = true
no_implicit_optional = true

# Enable gradually - add these when the codebase is annotated
# disallow_untyped_defs = true
# disallow_any_generics = true
# strict = true

# Third-party libraries without type stubs
[[tool.mypy.overrides]]
module = [
    "some_untyped_library.*",
    "another_library.*",
]
ignore_missing_imports = true

note

mypy with --strict is aggressive. It requires annotations on every function, disallows Any, and enforces correct generic usage. For a new codebase, start with basic annotations on public APIs and warn_return_any = true. Add disallow_untyped_defs = true when the team is comfortable. Save --strict for libraries that other code depends on, where type safety is critical.

Part 6 - The Complete Developer Workflow

In practice, your editor runs Black on save and Ruff in the background continuously. The workflow above represents what happens at commit time via pre-commit hooks (covered in the next lesson).

Part 7 - Tool Comparison

Tool	Category	Speed	Replaces	Config	Verdict
Ruff	Linter + Formatter	Rust, very fast	flake8, isort, pyupgrade, pydocstyle + plugins	pyproject.toml	Primary choice for linting
flake8	Linter	Python, moderate	-	setup.cfg / .flake8	Superseded by Ruff
pylint	Linter	Python, slow	-	.pylintrc	More rules than Ruff; much slower; use only for legacy projects
Black	Formatter	Fast	autopep8, yapf	pyproject.toml (minimal)	Standard formatter; no debate
isort	Import sorter	Fast	Manual sorting	pyproject.toml / .isort.cfg	Replaced by Ruff I rules
mypy	Type checker	Moderate (with cache)	-	pyproject.toml	Standard type checker
pyright	Type checker	Fast (Node.js/Rust)	-	pyrightconfig.json	VS Code default; stricter than mypy

A Minimal Production pyproject.toml

This is the configuration a new Python project should start with:

[tool.ruff]
target-version = "py311"
line-length = 88

[tool.ruff.lint]
select = ["E", "W", "F", "I", "N", "UP", "B", "RET", "SIM"]
ignore = ["E501"]
fixable = ["E", "W", "F", "I", "UP", "RET", "SIM"]

[tool.ruff.lint.per-file-ignores]
"tests/**/*.py" = ["S101", "ANN"]

[tool.black]
line-length = 88
target-version = ["py311"]

[tool.mypy]
python_version = "3.11"
warn_return_any = true
warn_unused_configs = true
no_implicit_optional = true

Graded Practice

Level 1 - Predict the Output

Look at the following code. Before running any tool, list all the Ruff rule violations you can identify (include the rule code if you know it):

import requests
import os
import sys
from typing import List,Dict

class user_account:
    Default_Role = 'admin'

    def GetUserName(self,id):
        response = requests.get(f'https://api.example.com/users/{id}')
        data=response.json()
        return data['name']

    def set_role(self,role,permissions:List=[]):
        self.role=role
        self.permissions=permissions

Show Answer

Violations found:

1. import requests - third-party import before stdlib imports; wrong order (I001 - isort)
2. import os and import sys - both unused (F401 × 2)
3. from typing import List,Dict - missing space after comma (E231); Dict unused (F401); List should be list in Python 3.9+ (UP006 × 2)
4. class user_account: - class name should be CapWords: should be UserAccount (N801)
5. Default_Role = 'admin' - class variable that is a constant should use SCREAMING_SNAKE_CASE only if it is truly a constant; mixing is confusing. Single quotes when project uses double (Black would fix). (Black: E501/quote style)
6. def GetUserName(self,id): - method name should be snake_case: should be get_user_name (N802); missing space after comma (E231)
7. data=response.json() - missing spaces around = (E225)
8. def set_role(self,role,permissions:List=[]): - mutable default argument [] - this is a classic Python bug where the list is shared across all calls (B006); missing spaces around = and after , (E225, E231)
9. self.role=role and self.permissions=permissions - missing spaces around = (E225 × 2)

The mutable default argument permissions: List = [] is the most dangerous: every call to set_role() that omits permissions shares the same list object. Appending to it in one call mutates it for all future calls that use the default. Fix: permissions: list | None = None, then if permissions is None: permissions = [] inside the function body.

Level 2 - Debug and Fix

The following mypy run produces three errors. Identify each error and provide the minimal fix:

# user_service.py

def get_username(user_id: int) -> str:
    users = {1: "Alice", 2: "Bob"}
    return users.get(user_id)


def send_notification(username: str, message: str) -> None:
    print(f"Sending to {username.upper()}: {message}")


def notify_user(user_id: int, message: str) -> None:
    name = get_username(user_id)
    send_notification(name, message)


results: list[str] = []
for uid in [1, 2, 3]:
    results.append(get_username(uid))

user_service.py:4: error: Incompatible return value type (got "str | None", expected "str")
user_service.py:12: error: Argument 1 to "send_notification" has incompatible type "str | None"; expected "str"

Show Answer

Error 1 - line 4: dict.get() returns str | None, but the return type says str

dict.get(key) returns ValueType | None because the key may not exist. The return type annotation -> str is wrong.

Fix option A - change return type to str | None:

def get_username(user_id: int) -> str | None:
    users = {1: "Alice", 2: "Bob"}
    return users.get(user_id)

Fix option B - use a default value:

def get_username(user_id: int) -> str:
    users = {1: "Alice", 2: "Bob"}
    return users.get(user_id, "Unknown")

Fix option C - raise on missing:

def get_username(user_id: int) -> str:
    users = {1: "Alice", 2: "Bob"}
    if user_id not in users:
        raise KeyError(f"No user with id {user_id}")
    return users[user_id]

Error 2 - line 12: name is str | None after fixing Error 1, but send_notification expects str

This cascades from Error 1. Once get_username correctly returns str | None, mypy knows name is str | None and rejects passing it to a function expecting str.

Fix - narrow the type before calling:

def notify_user(user_id: int, message: str) -> None:
    name = get_username(user_id)
    if name is None:
        raise ValueError(f"Cannot notify: user {user_id} not found")
    send_notification(name, message)

Or use the default-value version of get_username (Fix option B above), which eliminates the None case entirely.

Note: the code on lines 15–17 in the for loop also has a latent error - get_username returns str | None but results: list[str] expects str. mypy would flag results.append(get_username(uid)) with: Argument 1 to "append" of "list" has incompatible type "str | None"; expected "str". The same narrowing fix applies.

Level 3 - Design Challenge

You are joining a team with a 50,000-line Python codebase. It has no linting or formatting configured. Running ruff check . produces 1,847 violations. Running mypy src/ produces 342 errors. Running black --check . shows 892 files would be reformatted.

Design a migration strategy that:

1. Does not block the team from shipping features during the migration
2. Applies Black formatting without causing a massive merge conflict crisis for open branches
3. Introduces Ruff rules incrementally - starting from least disruptive to most
4. Introduces mypy incrementally - without requiring all 50,000 lines to be annotated first
5. Prevents regressions (new code does not re-introduce violations already fixed)

Justify your choices.

Show Answer

Phase 0 - Preparation (Day 1, 1 engineer, no team disruption)

Communicate the plan to the team. Create a dedicated branch for Phase 1. Agree on a code freeze window (e.g., Friday afternoon) for the formatting commit.

Phase 1 - Black formatting commit (Day 1–2)

The Black commit must happen on main with all open branches rebased afterward. The alternative - gradual formatting - is worse because every merged branch re-introduces unformatted code.

Strategy:

1. Create branch chore/black-formatting.
2. Run black ..
3. Commit with message: chore: apply Black formatting to all Python files.
4. Merge to main during low-activity window.
5. Communicate: all open branches must git rebase main and resolve conflicts (Black conflicts are mechanical - accept Black's version everywhere).
6. Add black --check . to CI immediately so no unformatted code can merge.

This is a one-time pain. Attempting gradual formatting makes the pain permanent.

Phase 2 - Ruff, safe rules only (Week 1)

Start with rules that are either auto-fixable or catch undeniable problems:

[tool.ruff.lint]
select = [
    "F",    # Pyflakes: undefined names, unused imports - these are bugs
    "E",    # pycodestyle: spacing - mostly auto-fixable
    "I",    # isort: import order - auto-fixable
]

Run ruff check --fix . on main. Commit. Add to CI.

This eliminates unused imports (real bugs), undefined names (real bugs), and spacing (cosmetic but auto-fixed). Zero team debate required.

Phase 3 - Ruff, opinion rules (Week 2–3)

Add UP (pyupgrade) and B (bugbear) categories. The B rules catch genuine bugs (mutable defaults, loop variable capture). Run ruff check --fix . for auto-fixable items; manually review the rest.

select = ["F", "E", "I", "UP", "B", "RET", "SIM"]

Apply per-file ignores for legacy files that have too many violations to fix immediately:

[tool.ruff.lint.per-file-ignores]
"src/legacy/**/*.py" = ["B", "UP"]

This lets you enforce rules on new code while not blocking on legacy.

Phase 4 - mypy, gradual (Week 3 onwards)

Do not enable --strict or disallow_untyped_defs initially. Start with:

[tool.mypy]
python_version = "3.11"
warn_return_any = true
no_implicit_optional = true
# Only check new/modified files strictly - use --follow-imports=silent for old code

Use # type: ignore strategically on legacy code paths. The key metric: mypy errors should trend down over time, and new code should not add new errors.

Enable disallow_untyped_defs = true per-module as each module's annotations are completed:

[[tool.mypy.overrides]]
module = ["myapp.services.*", "myapp.models.*"]
disallow_untyped_defs = true

Phase 5 - Preventing regressions

Add a pre-commit hook (see next lesson) that runs:

• ruff check --fix on staged files
• black --check on staged files
• mypy on staged files (initially just the changed files)

New code is always linted and typed. Legacy code is fixed opportunistically during normal development.

Key justification: The worst migration strategy is "fix everything in one PR." The second worst is "never fix it and let it drift." The correct strategy is: format in one commit (unavoidable), lint incrementally (per-rule, per-module), type-check gradually (per-module, not per-file). Progress is measurable: track violation counts and error counts weekly.

Key Takeaways

• Linting reports violations; formatting rewrites code deterministically. Both are necessary. They solve different problems.
• Ruff is the modern choice for Python linting. Written in Rust, it replaces flake8, isort, pyupgrade, and dozens of plugins in one tool. Start with E, F, I, UP, B rules and expand gradually.
• Black formats code to a canonical style with almost no configuration. The "no debates" philosophy is its core value. Configure your editor to run Black on save.
• isort sorts imports into stdlib / third-party / local groups. Use Ruff's I rules for import sorting instead of a separate isort installation; configure profile = "black" if you do use standalone isort.
• mypy performs static type checking at analysis time. Start with warn_return_any and no_implicit_optional; add disallow_untyped_defs per module as annotations are added. Do not go --strict immediately on a large codebase.
• # noqa: RULE suppresses a specific lint violation on one line. Always include the specific rule code, never use bare # noqa, and treat each one as debt to be revisited.
• The developer workflow is: edit → ruff check --fix→ black→ mypy→ pytest→ commit. In practice, your editor automates the first two steps, pre-commit hooks (Lesson 07) automate the rest.
• Configure all tools in pyproject.toml - one file, one place. No .flake8, no setup.cfg, no .isort.cfg. Modern tooling all reads pyproject.toml.
• A migration strategy for legacy codebases: Black first (one commit, everyone rebases), then Ruff safe rules (auto-fixed), then Ruff opinion rules (per-module ignores for legacy), then mypy gradual (per-module strict enables). Regressions are prevented by pre-commit hooks enforcing the new standard on all new code.