Module 01: ML Foundations
12 lessons | ~4 hours | The bedrock of every ML system you will ever build
Before you train your first model, before you reach for a GPU, before you open a Jupyter notebook - you need to understand what machine learning actually is, what makes it work, and what makes it fail.
This module is not a soft introduction. It is the conceptual engineering infrastructure that experienced ML engineers carry in their heads every time they debug a model, design an experiment, or review a colleague's approach. Skip these foundations and you will write code that runs but doesn't work, train models that fit but don't generalize, and ship systems that perform great in offline evaluation and fail silently in production.
Module Map
Lesson Map - What Each Lesson Unlocks
| Lesson | Core Concept | What It Unlocks |
|---|---|---|
| 01 - What is ML? | Optimization / compression / approximation view | You can reason about ML systems, not just use them |
| 02 - Learning Paradigms | Supervised, unsupervised, RL, self-supervised | You can pick the right paradigm before writing any code |
| 03 - ML Workflow | End-to-end pipeline from problem to monitoring | You stop treating model training as "the work" |
| 04 - Data Representation | Feature spaces, curse of dimensionality | You understand why preprocessing choices matter architecturally |
| 05 - Bias-Variance | MSE decomposition, double descent | You can diagnose model failures from learning curves |
| 06 - Generalization | Overfitting, regularization | You know why your model does not deploy as well as it trains |
| 07 - Classification Metrics | Precision, recall, AUC, F1 | You choose the right metric for the real business objective |
| 08 - Regression Metrics | MAE, RMSE, R², MAPE | You diagnose regression model failures quantitatively |
| 09 - Cross-Validation | k-Fold, stratified, time-series CV | You produce reliable estimates of generalization performance |
| 10 - Split Strategy | Leakage, group splits, temporal splits | You stop accidentally leaking the future into training |
| 11 - Probabilistic View | MLE, MAP, calibration | You understand why loss functions are what they are |
| 12 - Statistical Learning Theory | PAC, VC dimension, sample complexity | You reason about how much data you need and why |
Why Foundations Matter Even If You Use AutoML
AutoML tools (Google Vertex AI AutoML, AWS AutoPilot, H2O.ai, AutoGluon) automate hyperparameter search, architecture search, and sometimes feature engineering. A reasonable engineer might ask: if the machine picks the model, why do I need to understand the foundations?
Here is why:
1. AutoML does not frame your problem. The single most important decision in any ML project is what you are optimizing for. AutoML cannot tell you whether to optimize for precision or recall, whether your class imbalance requires special handling, or whether the metric you are optimizing is actually the metric that matters to the business. Lessons 07, 08, and the probabilistic framing in Lesson 11 are the only things that help here.
2. AutoML does not detect data leakage. Data leakage - where information from the future or from the test set bleeds into training - is the single most common source of falsely high offline performance in production ML systems. AutoML will happily train on leaked data and report excellent results. You will discover the problem at deployment. Lessons 09 and 10 cover this in depth.
3. AutoML does not tell you when you have enough data. Statistical learning theory (Lesson 12) gives you the tools to reason about whether 10,000 labeled examples is enough or whether you need 1,000,000. AutoML will train on whatever you give it.
4. AutoML does not explain why the model fails. When a production model starts degrading - and it will - you need the bias-variance framework (Lesson 05), the generalization framework (Lesson 06), and the right evaluation metrics (Lessons 07–08) to diagnose the failure and fix it.
AutoML is a search tool. These foundations are the engineering judgment you need to use it correctly.
Prerequisites
| Prerequisite | Where to get it |
|---|---|
| Python - comfortable with numpy arrays, list comprehensions | Any Python basics course |
| Basic statistics - mean, variance, probability | High school / first-year stats |
| Calculus - what a derivative is (not how to compute complex ones) | Khan Academy Calculus |
| Linear algebra basics - vectors, matrices | Math for AI → Module 01 |
You do not need prior ML experience. This module assumes you have seen sklearn examples but could not explain why they work.
Learning Path
This module is the foundation for everything else in the ML curriculum:
ML Foundations (this module)
│
├── Linear Models (Module 02) - uses bias-variance, ERM, regularization
├── Tree-Based Models (Module 03) - uses generalization, feature spaces
├── Neural Networks (Module 04) - uses probabilistic view, gradient descent
├── Feature Engineering (Module 05) - uses representation, encoding
└── ML System Design (Module 09) - uses workflow, evaluation, split strategy
Complete this module before moving to any other. The concepts here will be referenced in every subsequent lesson.
How to Use This Module
Each lesson follows the same structure:
- A production scenario - a real failure mode or system challenge that motivates the concept
- Core concepts - rigorous treatment with math and code
- Python examples - runnable code with sklearn, numpy, and sometimes PyTorch
- Engineering perspective - what this means when you are building systems, not just running experiments
- Interview Q&A - 5 questions with detailed answers at the level of a senior ML engineer interview
Work through the lessons in order. Lessons 05 and 06 build on each other. Lessons 09 and 10 are a pair. Lessons 11 and 12 are the theoretical capstone and can be read after the rest.
Start with Lesson 01 - What is Machine Learning? →