Skip to main content

167 docs tagged with "python-intermediate"

View all tags

Asyncio and Async/Await

Master Python's async/await model - coroutines, tasks, gather, event loop, async context managers, and building high-performance I/O-bound applications.

Building an Async API Service

Apply asyncio in production - async FastAPI routes, background tasks, async database access, connection pools, and handling concurrency in a real API service.

Bytecode Inspection - Inside the code Object

Understand Python bytecode and the code object at engineering depth - all co_ attributes explained, how .pyc files work, reading bytecode with marshal, the line number table, closures in bytecode, and practical uses in debuggers and test frameworks.

Code Coverage - Measuring What You Test (and What You Miss)

Master code coverage at engineering depth - line vs branch vs condition coverage, coverage.py internals with sys.settrace, pytest-cov, .coveragerc configuration, pragma no cover, coverage in CI, and mutation testing with mutmut to find tests that pass but don't catch bugs.

Database Migrations with Alembic

Manage database schema changes safely using Alembic - auto-generated migrations, upgrade/downgrade, migration environments, and production deployment strategies.

Database Module Projects

Three production-style database engineering projects - a CRUD app with proper data layer, a transaction-safe service, and a full ORM-backed API.

Dataclasses - Code Generation, Immutability, and Production Patterns

Master Python's @dataclass decorator at engineering depth - what it generates, field() and default_factory, frozen=True for immutability, __post_init__ for validation, ClassVar vs InitVar, inheritance with dataclasses, ordering, and production patterns in FastAPI and config systems.

Decorators - Wrapping Callables at Engineering Depth

Master Python decorators at full engineering depth - functools.wraps, decorator factories with three-level nesting, class-based decorators, stacking order, production patterns (timing, retry, caching, rate limiting), and how FastAPI/Flask route decorators work under the hood.

Disassembly with dis - Reading CPython Bytecode

Master Python bytecode disassembly with the dis module at engineering depth - reading disassembly output, key opcodes explained, value stack evolution, comparing equivalent Python patterns at the instruction level, and practical performance insights.

Flask - Building REST APIs the Right Way

Master Flask at engineering depth - application factory pattern, request context proxies, routing, Blueprints, error handlers, testing with test_client, configuration management, and the extension ecosystem for building production-grade REST APIs.

FP Module Projects - Engineering Challenges

Three hands-on functional programming engineering projects for the Python Intermediate module. Build real systems using decorators, generators, closures, composition, and pure functions.

HTTP Deep Dive - What Actually Travels Over the Wire

Master HTTP/1.1 at the byte level - request/response wire format, method semantics, status code families, critical headers, connection pooling, the requests and httpx libraries, HTTP/2 multiplexing, and why every production client needs explicit timeouts.

Immutability Strategies - Tuples, Frozen Dataclasses, and Value Objects

Master Python's immutability toolkit at engineering depth - mutable vs immutable types, shallow vs deep immutability, namedtuple, frozen dataclasses, frozenset, MappingProxyType, and the replace/copy pattern for functional state updates. Covers DDD value objects and Redux-style state in Python.

Indexing and Query Optimization

Understand database indexes from the ground up - B-tree internals, query planning, EXPLAIN ANALYZE, composite indexes, and when indexes hurt performance.

JSON Serialization - Production-Grade Encoding and Decoding

Master JSON serialization in Python at engineering depth - custom encoders, datetime/Decimal/UUID handling, orjson and msgspec for high-throughput APIs, NDJSON streaming, content negotiation, and why float precision silently destroys financial data.

Linting and Formatting - Ruff, Black, isort, and mypy

Master Python code quality tooling at engineering depth - Ruff's rule categories, Black's opinionated formatting, isort profiles, mypy static type checking, pyproject.toml configuration, and how to wire all tools into a coherent developer workflow.

Locks, Semaphores, and Synchronization

Master Python synchronization primitives - Lock, RLock, Semaphore, Event, Condition, and Barrier - and when to use each to build correct concurrent systems.

Memory Profiling - tracemalloc, memory_profiler, objgraph, and pympler

Profile and debug Python memory usage at engineering depth - sys.getsizeof shallow vs deep size, tracemalloc snapshots and leak detection, memory_profiler line-by-line analysis, objgraph retention paths, pympler recursive sizing, and practical workflows for diagnosing real-world memory leaks.

Middleware - Wrapping Every Request and Response

Master middleware at engineering depth - WSGI vs ASGI middleware, the onion model, request ID propagation, timing, structured logging, CORS, rate limiting with Redis, JWT authentication, and when to use middleware vs dependency injection.

Module 01 - Object-Oriented Programming Overview

Master Python's object model at engineering depth - classes, instances, dunder methods, encapsulation, inheritance, MRO, composition, abstract base classes, dataclasses, SOLID principles, and production design patterns.

Module 02 - Functional Programming Overview

Master Python's functional programming model at engineering depth - lambdas, map/filter/reduce, generators, iterators, decorators, closures, pure functions, immutability, functools, and partial application and currying.

Module 03 - Python Internals Overview

Understand CPython's implementation details at engineering depth - bytecode, the eval loop, the GIL, reference counting, garbage collection, memory profiling, sys/inspect, and the import system.

Module 04 - Testing and Quality Overview

Build production-grade test suites at engineering depth - unittest, pytest, mocking, TDD, code coverage, linting, and pre-commit hooks that enforce quality at every commit.

Module 04 Projects - Testing and Quality

Overview of the two hands-on engineering projects for the Testing and Quality module - a full pytest suite for a banking system and a complete CI quality pipeline setup.

Module 06 - APIs and Web Basics

Master HTTP at the wire level, REST design principles, Flask, FastAPI, request/response lifecycle, middleware, JSON serialization, and Pydantic validation - the complete engineering foundation for building production web APIs in Python.

Module 06 Projects - Overview

Two production-focused projects for Module 06 - a fully tested Task Management REST API with FastAPI and SQLAlchemy, and a local deployment stack with Docker, Nginx, PostgreSQL, and Alembic migrations.

Module 07 - Databases Overview

Master database engineering in Python - SQL, SQLite, PostgreSQL, transactions, indexing, SQLAlchemy ORM, and migrations.

Module 08 - Concurrency Overview

Master concurrency in Python - threading, multiprocessing, asyncio, event loops, race conditions, locks, and building production async systems.

MRO - Method Resolution Order and the C3 Linearisation Algorithm

Understand Python's Method Resolution Order at engineering depth - the diamond problem, C3 linearisation step by step, how super() traverses the MRO (not just "calls parent"), mixin patterns that depend on MRO, Django/Flask examples, and MRO failure cases.

Multiprocessing in Python

Bypass the GIL with multiprocessing - Process, Pool, shared memory, queues, pipes, and when to use processes instead of threads.

ORM with SQLAlchemy

Master SQLAlchemy - the declarative ORM, session management, relationships, lazy vs eager loading, and building production data access layers.

Packaging and Environments - Module Overview

Master Python packaging and environments at full engineering depth - virtual environments, pip and lockfiles, pyproject.toml, Poetry, semantic versioning, and publishing to PyPI for production-grade projects.

Packaging Projects - Overview

Overview of hands-on projects for Module 05 - Packaging and Environments. Build, test, version, and publish a real Python utility package from scratch.

Partial Application and Currying - functools.partial, operator, and Function Pipelines

Master partial application and currying at engineering depth - functools.partial internals, inspecting partial objects, the distinction between partial application and currying, implementing currying in Python, the operator module as curried-style operations, function composition with reduce, and real-world usage in Django ORM, sorted(), and data pipelines.

pip and requirements - Dependency Management in Practice

Master pip and requirements files at full engineering depth - dependency resolution, version specifiers, pip-tools lockfiles, layered requirements, hash verification, supply-chain security, and private package indexes for production workflows.

PostgreSQL with Python

Connect Python to PostgreSQL using psycopg2 and psycopg3 - connection pooling, parameterized queries, JSONB, arrays, and production connection management.

Pre-Commit Hooks - Automate Quality Gates Before Every Commit

Master the pre-commit framework at engineering depth - Git hook mechanics, .pre-commit-config.yaml structure, building production hook pipelines with ruff, black, mypy, detect-secrets, and pytest, CI integration, team adoption strategy, and hook performance tuning.

Project 01 - Banking System Simulator

Build a banking system using Python OOP. Covers inheritance, @property validation, @classmethod factory methods, __repr__, overdraft protection, and transaction history.

Project 01 - Bytecode Visualizer

Build a Python bytecode analysis tool using the ast module, dis.get_instructions(), and CPython's value stack model. Parse source to AST, disassemble to bytecode, simulate stack evolution, and compare two implementations side by side.

Project 01 - Custom Decorator Library

Build a production-quality Python decorator library with retry, rate limiting, timeout, argument validation, and structured call logging. Covers closures, functools.wraps, inspect.signature, and decorator composition.

Project 01 - Full Test Suite for a Banking System

Build a production-quality pytest test suite for the BankAccount and SavingsAccount system - fixtures, parametrized edge cases, mocked external dependencies, integration tests, Hypothesis property-based tests, and 90% branch coverage enforcement.

Project 01 - Publish an Internal Utility Package

Build, test, version, and publish pyutils-engineersofai - a typed Python utility library with src/ layout, hatchling build backend, full pytest coverage, CHANGELOG, and GitLab CI pipeline that publishes on v* tags.

Project 01 - Task Management REST API

Build a production-quality Task Management REST API with FastAPI, Pydantic v2, SQLAlchemy, pytest TestClient, RFC 7807 error responses, request ID middleware, and Docker - full CRUD, pagination, filtering, and OpenAPI docs.

Project 02 - CI Quality Pipeline Setup

Configure a complete Python quality pipeline from scratch - pre-commit hooks, pyproject.toml tool configuration, GitLab CI four-stage pipeline, tox/nox multi-version testing, Makefile targets, coverage gate, and JUnit test report artifacts.

Project 02 - Lazy Evaluation Pipeline

Build a lazy data processing pipeline in Python that handles datasets larger than RAM. Covers generators, generator chaining, lazy evaluation, immutable pipeline objects, and streaming architecture.

Project 02 - Library Management System

Build a library management system using Python OOP. Covers __repr__, __eq__, __hash__, Abstract Base Classes, checkout systems with date tracking, overdue detection, and fine calculation.

Project 02 - Local Deployment Setup

Configure a production-like local deployment stack for the Task Management API - multi-stage Dockerfile, Docker Compose with Nginx and PostgreSQL, Alembic migrations, Pydantic BaseSettings, health checks, graceful shutdown, and a Makefile for operations.

Project 02 - Mini Profiler Tool

Build a production-quality Python profiler using sys.setprofile(), tracemalloc, and call tree construction. Supports context manager and decorator usage, formatted report tables, run comparison, and JSON export for CI integration.

Project 03 - Chess Engine (OOP Version)

Build a working chess engine in Python using ABCs, dunders, dataclasses, and composition. Covers Piece ABC, concrete piece classes, Board with __getitem__/__setitem__, Move dataclass, check detection, and algebraic notation.

Project 03 - Functional Data Processor

Build a functional-style data transformation system in Python using function composition, pipe, partial, singledispatch, and the Result monad pattern. All functions are pure; error handling avoids exceptions.

Project: Async Data Aggregation API

Build an async FastAPI service that aggregates data from multiple external APIs concurrently - with caching, rate limiting, circuit breaking, and background refresh.

Project: Concurrent Web Scraper

Build a production-grade concurrent web scraper using ThreadPoolExecutor and asyncio - rate limiting, retry logic, robots.txt compliance, and structured output.

Project: Library Management System

Build a command-line library management system backed by SQLite - books, members, loans, with proper SQL, transactions, and a clean data access layer.

Publishing Packages - From Source to PyPI

Master Python package publishing at engineering depth - sdist vs wheel formats, build backends, TestPyPI workflow, twine and Poetry publishing, API tokens, private registries, and automated CI/CD release pipelines.

pyproject.toml - The Modern Python Project Standard

Master pyproject.toml at full engineering depth - PEP 517/518/621 build system specification, build backends, the full project table, optional dependencies, entry points, tool configuration, src layout, dynamic versioning, and building distribution artifacts.

pytest - The Industry-Standard Test Framework

Master pytest at full engineering depth - assertion rewriting via AST transformation, fixtures with scope, conftest.py, parametrize, monkeypatch, capsys, built-in marks, essential plugins, and pyproject.toml configuration for production test suites.

Race Conditions and Thread Safety

Understand race conditions at the CPU level - why they happen, how to detect them, and how to write thread-safe Python code that behaves correctly under concurrency.

REST Principles - Designing APIs That Don't Break Clients

Master REST at engineering depth - Roy Fielding's six constraints, uniform interface, URL design, HTTP method semantics, status codes, pagination patterns, versioning strategies, RFC 7807 error format, and the Richardson Maturity Model.

SQLite with Python

Use SQLite for development, testing, and embedded applications - the sqlite3 module, connection management, cursors, parameterized queries, and row factories.

sys and inspect - Runtime Introspection at Engineering Depth

Master the sys and inspect modules at engineering depth - sys.argv, sys.path, sys.modules cache, sys.settrace, sys._getframe, inspect.signature with all parameter kinds, inspect.getsource, inspect.stack, and how FastAPI, pytest, and click use these modules to build their core features.

The Event Loop Explained

Understand Python's asyncio event loop at engineering depth - how it works, selectors, callbacks, handles, and debugging event loop issues in production.

Threading in Python

Master Python threading - Thread creation, daemon threads, thread lifecycle, sharing state, the GIL's real impact, and when threading actually helps performance.

Transactions and Data Integrity

Master database transactions in Python - ACID properties, isolation levels, deadlocks, savepoints, and building transaction-safe data layers.

unittest - The Standard Library Test Framework

Master Python's unittest framework at engineering depth - TestCase lifecycle, all assertion methods, assertRaises as context manager, setUp/tearDown/setUpClass, unittest.mock.patch, TestSuite, skip decorators, and subtests for parametrised testing.

venv and virtualenv - Python Environment Isolation

Master Python virtual environments at full engineering depth - how venv works at the filesystem level, PATH manipulation, pyvenv.cfg, pyenv for Python version management, and why Docker containers are not a substitute for virtual environments.