01Module 03 - Tree Models and EnsemblesMaster decision trees and ensemble methods from first principles - the model family that dominates tabular ML competitions and powers production fraud, pricing, and ranking systems worldwide.02Decision Trees InternalsDeep dive into decision tree internals - recursive binary splitting, CART, Gini and entropy impurity, pruning, and a full from-scratch NumPy implementation for classification and regression.03Information Gain, Gini Impurity, and EntropyA deep dive into how decision trees choose splits - Shannon entropy, information gain, Gini impurity, gain ratio, regression variance reduction, and the multi-valued feature bias every practitioner must understand.04Pruning and Depth ControlHow to prevent decision tree overfitting through pre-pruning parameters, cost-complexity post-pruning, weakest-link pruning, MDL principle, and production-grade tuning strategies.05Random ForestsMaster Random Forests from first principles - bagging variance reduction math, feature randomization, OOB error estimation, Extra-Trees, bias-variance decomposition, MDI vs permutation importance, and production deployment patterns.06Gradient Boosting From ScratchUnderstand gradient boosting from first principles - additive models, functional gradient descent, pseudo-residuals for any loss function, shrinkage, stochastic boosting, and bias-variance tradeoffs versus Random Forest.07XGBoost Deep DiveMaster XGBoost internals - the 7 innovations over vanilla gradient boosting, optimal leaf weights, gain calculation, hyperparameter tuning, and production deployment with ONNX and GPU training.08LightGBM and CatBoostMaster LightGBM's GOSS and EFB algorithms, CatBoost's ordered target statistics, and learn when to choose each framework for large-scale tabular machine learning.09Feature Importance and SHAPMaster all three feature importance types, TreeSHAP for exact Shapley values, SHAP interaction values, feature selection with SHAP, data leakage detection, fairness analysis, and production importance drift monitoring.10Stacking and BlendingMaster stacking and blending ensemble techniques - out-of-fold meta-learning, data leakage prevention, model diversity, snapshot ensembling, temporal ensembling, Kaggle competition patterns, and production deployment tradeoffs.
