01Module 12 - LLMOps PipelinesOperationalize LLM-based systems - prompt management, evaluation pipelines, observability, RAG operations, and fine-tuning infrastructure.02Fine-Tuning OpsOperationalize LLM fine-tuning at scale - data pipelines, LoRA adapter management, adapter registries, and serving 50 customer-specific adapters efficiently.03Prompt ManagementTreat prompts as production artifacts - versioning, registry design, testing frameworks, A/B testing prompts, automated optimization with DSPy, and prompt governance.04LLM Evaluation PipelinesBuild automated evaluation pipelines for LLM systems - LLM-as-judge, RAGAS for RAG systems, trajectory evaluation for agents, regression testing, and eval dataset curation.05RAG Pipeline OpsOperate RAG pipelines in production - index refresh strategies, chunk strategy updates, embedding drift detection, vector database monitoring, and quality tracking.06Token Cost MonitoringMonitor and control LLM API costs in production - cost-per-request dashboards, budget alerts, token efficiency optimization, cost attribution by feature and user, and anomaly detection.
