01Module 12: State Space ModelsA complete map of State Space Models - from the quadratic attention bottleneck to Mamba's selective recurrence, hybrid architectures, and production deployment.02Limitations of Attention at ScaleWhy the quadratic complexity of self-attention creates real production bottlenecks - memory, latency, and cost - and why sparse attention approximations only partially solve the problem.03State Space Model FoundationsHow control theory's state space models became a competitive sequence modeling architecture - continuous-time SSMs, the S4 paper, HiPPO initialization, and the convolutional/recurrent duality.04Mamba - Selective State Space ModelsHow Mamba's input-dependent SSM parameters, hardware-aware parallel scan, and selective gating mechanism achieved linear-time sequence modeling competitive with transformers.05Mamba vs Transformer - When Each WinsA rigorous benchmark comparison: perplexity, throughput, recall tasks, in-context learning, and the fundamental trade-off between compressed state and full context access.06Hybrid Architectures - Jamba and BeyondHow combining attention and Mamba layers creates models that outperform pure architectures - Jamba's design, the attention-to-Mamba ratio, MoE integration, and the emerging hybrid landscape.07When to Use SSMs in ProductionA practical deployment guide: use cases where SSMs win, the streaming inference pattern, model availability on HuggingFace, fine-tuning SSMs, and a forward-looking outlook.
