Taylor Series and Approximations - The Mathematics Behind Gradient Descent and Newton's Method
A deep engineering dive into Taylor expansions, why gradient descent uses first-order approximations, how Newton's method uses curvature, quasi-Newton methods, and their practical implications for ML optimization.