12 docs tagged with "linear-algebra"

Dot Products and Projections - The Math Behind Attention

A deep engineering dive into dot products, orthogonality, vector projection, Gram-Schmidt orthogonalization, and least squares - the mathematical heart of the transformer attention mechanism.

Eigenvalues and Eigenvectors - How PageRank Ranks the Internet

A deep engineering dive into eigenvalues, eigenvectors, and eigendecomposition - the mathematics behind PCA, PageRank, spectral clustering, and graph neural networks.

Linear Algebra for Machine Learning - Module Overview

A complete module map showing how vectors, matrices, eigenvalues, SVD, and tensors connect to every major ML algorithm - from attention to PCA to backpropagation.

Linear Algebra in NumPy - A Complete Engineering Reference

A complete engineering reference for NumPy linear algebra - np.linalg module, solving systems, decompositions, performance tips, numerical stability, and PyTorch torch.linalg equivalents.

Linear Transformations - The Geometry of Neural Network Layers

A deep engineering dive into linear maps, kernel (null space), image (column space), rank-nullity theorem, and change of basis - the geometry behind every neural network layer.

Matrix Operations - The Engine of the Neural Network Forward Pass

A deep engineering dive into matrix multiplication, transpose, inverse, rank, and determinant - and how they power attention mechanisms, backpropagation, and neural network layers.

Norms and Distance Metrics - The Geometry of Regularization

A deep engineering dive into L1, L2, L∞, Frobenius, and nuclear norms - and how the geometry of different norms determines which model weights go to zero in Lasso vs Ridge regularization.

Numerical Linear Algebra - Condition Numbers, Solvers, and Backprop Stability

Engineering guide to condition numbers, ill-conditioned matrices, LU/QR/Cholesky factorizations, why you should never invert a matrix, and the numerical stability of neural network backpropagation.

PCA from Linear Algebra - Eigendecomposition of Covariance Matrices

A complete derivation of Principal Component Analysis from first principles - covariance matrices, eigendecomposition, explained variance, and PCA via SVD - the way sklearn actually does it.

SVD and Matrix Decompositions - The Netflix Algorithm and JPEG Compression

A deep engineering dive into Singular Value Decomposition, LU, QR, and Cholesky decompositions - the mathematical tools behind recommender systems, image compression, and numerically stable ML computations.

Tensors for Deep Learning - From Matrices to Multi-Dimensional Arrays

A deep engineering dive into tensors as generalizations of matrices - shapes, axes, contractions, Einstein summation, broadcasting, and vectorization - the computational substrate of every deep learning model.

Vectors and Vector Spaces - The Language of Embeddings

A deep engineering dive into vectors, vector spaces, norms, and inner products - the mathematical foundation behind embeddings, cosine similarity, and KNN in ML systems.