Cascade Funnel Architecture - Interactive Visualization

The cascade funnel is the standard architecture for industrial recommendation and search systems. It solves a core constraint: you have 100 million items but only 100ms to return a ranked list. The solution is a sequence of increasingly expensive but increasingly accurate models applied to a shrinking candidate set. Stage 1 (retrieval) uses a cheap ANN or BM25 lookup to pull 100k candidates in 5ms. Stage 2 (pre-ranking) scores 10k candidates with a lightweight model. Stage 3 (ranking) applies your expensive neural model to just 500 candidates. Stage 4 (re-ranking) applies business rules and diversity constraints to the final 50. The total cost is orders of magnitude lower than ranking all candidates with the full model.

• Retrieval: approximate nearest neighbor search (FAISS, ScaNN) or BM25 - retrieves 100k candidates in milliseconds
• Pre-ranking: shallow model (logistic regression or small MLP) applied to 10k candidates - eliminates obvious non-relevant items
• Ranking: your best neural model applied to 500 candidates - most of the quality budget spent here
• Re-ranking: business logic - deduplication, diversity injection, sponsored item insertion - applied to top 50

Part of the EngineersOfAI Interactive 3D - free interactive visualizations covering every major concept in machine learning and AI engineering. Hover any element for a plain-English explanation. No code required.