CASCADE: Case-Based Continual Adaptation for Large Language Models During Deployment
:::info Stub — Full Engineering Breakdown Coming This paper has a linked code implementation and was featured on Hugging Face Papers with 1 upvotes. A full breakdown with production viability rating, implementation notes, and honest limitations is being written. Subscribe to AI Letters → :::
| Authors | Siyuan Guo et al. |
| Year | 2026 |
| HF Upvotes | 1 |
| arXiv | 2605.06702 |
| Download | |
| Code | https://github.com/guosyjlu/CASCADE |
Abstract
Large language models (LLMs) have become a central foundation of modern artificial intelligence, yet their lifecycle remains constrained by a rigid separation between training and deployment, after which learning effectively ceases. This limitation contrasts with natural intelligence, which continually adapts through interaction with its environment. In this paper, we formalise deployment-time learning (DTL) as the third stage in the LLM lifecycle that enables LLM agents to improve from experience during deployment without modifying model parameters. We present CASCADE (CASe-based Continual Adaptation during DEployment), a general and principled framework that equips LLM agents with an explicit, evolving episodic memory. CASCADE formulates experience reuse as a contextual bandit problem, enabling principled exploration-exploitation trade-offs and establishing no-regret guarantees over long-term interactions. This design allows agents to accumulate, select, and refine task-relevant cases, transforming past experience into actionable knowledge. Across 16 diverse tasks spanning medical diagnosis, legal analysis, code generation, web search, tool use, and embodied interaction, CASCADE improves macro-averaged success rate by 20.9% over zero-shot prompting while consistently outperforming gradient-based and memory-based baselines. By reframing deployment as an adaptive learning process, this work establishes a foundation for continually improving AI systems.
Engineering Breakdown
Plain English
This paper introduces CASCADE, a framework that lets LLMs learn and improve from user interactions during deployment without retraining or updating model weights. Instead of the traditional train-once-then-freeze approach, CASCADE maintains an evolving episodic memory of past cases and uses contextual bandits to decide which cached examples to retrieve and use as in-context learning material, enabling the model to adapt to new patterns and user preferences in real time.
Key Engineering Insight
The core innovation is treating deployment-time learning as a retrieval and ranking problem rather than a parameter update problem—CASCADE builds an explicit memory of interaction cases and learns which examples to surface in prompts to improve future responses, sidestepping the computational cost and complexity of continual fine-tuning.
Why It Matters for Engineers
For production systems, this solves a critical gap: model behavior drift when deployed to new user populations or domains, without requiring expensive retraining pipelines or model updates that risk breaking existing functionality. It's a practical way to let deployed LLMs adapt to local patterns and user feedback in hours, not weeks, while maintaining safety and reproducibility.
Research Context
Prior work separated LLM lifecycle into training and deployment with no learning in-between, leaving deployed models unable to improve from real-world usage. CASCADE formalizes deployment-time learning as a distinct stage and provides a principled mechanism using case-based reasoning and contextual bandits. This bridges the gap between static deployment and continuous adaptation, enabling LLM agents to behave more like human intelligence that learns through experience rather than retraining.
:::tip Subscribe Get weekly breakdowns of papers like this in AI Letters - the newsletter for engineers building production AI systems. :::