01Module 04: Synthetic DataLearn to generate, filter, and use synthetic training data at scale - from Self-Instruct bootstrapping to Evol-Instruct complexity evolution, distillation datasets, and RAG evaluation corpora.02Why Synthetic DataUnderstand why synthetic data has become central to AI engineering - the labeled data bottleneck, privacy constraints, rare events, LLMs as generators, landmark case studies, and when synthetic beats real.03LLM as Data GeneratorUse frontier LLMs to generate high-quality instruction-following, reasoning, and preference datasets - sampling strategies, diversity maximization, and quality vs. quantity tradeoffs.04Self-InstructHow the Self-Instruct paper bootstrapped instruction-following datasets from a tiny seed set using GPT-3, enabling the Alpaca era of aligned models - and how to implement it today.05Evol-InstructEvol-Instruct: systematically evolving instruction datasets to create complex, diverse training data that produces stronger instruction-following models - the technique behind WizardLM and WizardCoder.06Distillation DatasetsBuilding distillation datasets: capturing frontier model knowledge, reasoning traces, and calibration into training data for smaller, efficient models - from Orca to Phi.07Data Quality and FilteringSystematic approaches to filtering synthetic data for quality, diversity, safety, and alignment - the layered pipeline that separates fine-tuned models that work from models that regress.08Synthetic Data for RAGGenerating question-answer pairs, evaluation datasets, and retrieval test cases from documents to build, evaluate, and systematically improve RAG systems.09Privacy and Ethics in Synthetic DataCopyright exposure, memorization risks, differential privacy, bias auditing, terms-of-service compliance, and the governance processes required for defensible synthetic data pipelines.
