01Module 01: LLMOps - OverviewAn overview of LLMOps - the engineering discipline for building, shipping, and operating production LLM applications reliably and at scale.02What is LLMOpsLLMOps defined - the operational discipline for managing LLM-powered applications in production, why it differs from MLOps, and the full lifecycle every AI engineering team must master.03Prompt VersioningTreating prompts as first-class code artifacts - versioning, branching, review gates, A/B testing, and rollback for production LLM prompts. Build a complete prompt registry from scratch.04Fine-Tuning PipelinesEnd-to-end fine-tuning pipeline engineering - from data collection and curation to training, evaluation, and deployment. When to fine-tune vs RAG vs prompt engineering, and how to build the pipeline that makes it repeatable and production-safe.05LLM CI/CDCI/CD pipelines for LLM applications - handling non-deterministic outputs with LLM-judge gates, canary deployments with quality monitoring, automated rollback triggers, and full GitHub Actions implementation.06Dataset Curation for Fine-TuningHow to build high-quality fine-tuning datasets - sourcing, deduplication, quality filtering, LLM-as-judge scoring, and a complete curation pipeline. Why 5K curated examples beat 500K raw ones.07Evaluation-Driven DevelopmentBuilding AI systems test-first - write evals before writing prompts. The EDD loop, eval strategies, golden dataset construction, LLM-as-judge calibration, and a full EvalSuite implementation ready for CI integration.08LLMOps PlatformsComprehensive guide to LLMOps platforms - LangSmith, Langfuse, W&B Weave, Arize Phoenix, Helicone, and PromptLayer. When to build vs buy, integration patterns, abstraction layers, and production-grade Python examples using the Anthropic SDK.
