Deep engineering guide to IEEE 754 floating-point, machine epsilon, catastrophic cancellation, float16/bfloat16/float32 in deep learning, and numerical stability techniques for production ML systems.
Engineering guide to iterative methods for linear systems - conjugate gradient, GMRES, preconditioning, and when iterative solvers beat direct methods in large-scale ML workloads.
Overview of numerical methods for AI - floating-point precision, linear solvers, automatic differentiation, sparse matrices, and why numerical stability determines whether your model trains or diverges.
Engineering guide to finite difference methods, central difference formulas, step size selection, truncation vs rounding error, and gradient checking to validate automatic differentiation implementations.
Engineering guide to numerical integration methods - quadrature rules, Monte Carlo integration, importance sampling, and applications in Bayesian inference, variational methods, and normalizing constants.
Engineering guide to condition numbers, ill-conditioned matrices, LU/QR/Cholesky factorizations, why you should never invert a matrix, and the numerical stability of neural network backpropagation.
Engineering guide to root-finding algorithms - bisection method, Newton-Raphson, secant method, convergence rates, and ML connections including learning rate scheduling and fixed-point iterations.
Engineering guide to sparse matrix storage formats (CSR, CSC, COO, LIL), sparse operations in SciPy, and why sparsity is fundamental to attention masks, graph adjacency matrices, and embedding tables in production ML.
