SenCache: Accelerating Diffusion Model Inference via Sensitivity-Aware Caching
| Authors | Yasaman Haghighi & Alexandre Alahi |
| Year | 2026 |
| Field | Computer Vision |
| arXiv | 2602.24208 |
| Download | |
| Categories | cs.CV, cs.LG |
Abstract
Diffusion models achieve state-of-the-art video generation quality, but their inference remains expensive due to the large number of sequential denoising steps. This has motivated a growing line of research on accelerating diffusion inference. Among training-free acceleration methods, caching reduces computation by reusing previously computed model outputs across timesteps. Existing caching methods rely on heuristic criteria to choose cache/reuse timesteps and require extensive tuning. We address this limitation with a principled sensitivity-aware caching framework. Specifically, we formalize the caching error through an analysis of the model output sensitivity to perturbations in the denoising inputs, i.e., the noisy latent and the timestep, and show that this sensitivity is a key predictor of caching error. Based on this analysis, we propose Sensitivity-Aware Caching (SenCache), a dynamic caching policy that adaptively selects caching timesteps on a per-sample basis. Our framework provides a theoretical basis for adaptive caching, explains why prior empirical heuristics can be partially effective, and extends them to a dynamic, sample-specific approach. Experiments on Wan 2.1, CogVideoX, and LTX-Video show that SenCache achieves better visual quality than existing caching methods under similar computational budgets.
Engineering Breakdown
Plain English
SenCache proposes a principled framework for accelerating diffusion model inference by intelligently caching and reusing computed outputs across denoising timesteps. The key insight is that not all timesteps contribute equally to the final output quality—by analyzing model sensitivity to input perturbations (both the noisy latent and timestep values), the authors can identify which timesteps are critical and which can safely reuse cached values from previous computations. This training-free approach eliminates the need for heuristic tuning that existing caching methods require, offering a more systematic way to trade off computation cost against generation quality in video and image synthesis tasks.
Core Technical Contribution
The paper's core novelty is formalizing diffusion model caching through sensitivity analysis rather than heuristics. The authors derive an analytical framework that quantifies how sensitive the model output is to perturbations in the denoising inputs—specifically measuring how much the model output changes when you perturb the noisy latent vector or the timestep embedding. This sensitivity metric becomes a principled criterion for deciding which timesteps can reuse cached outputs from neighboring timesteps without significant quality degradation. Unlike prior work that uses fixed or manually-tuned reuse schedules, this approach is adaptive and theoretically grounded.
How It Works
The method operates in two phases: first, it analyzes the diffusion model's behavior by computing sensitivity scores that measure output perturbation response at each timestep. For each timestep t, the authors compute how much the model output changes when small perturbations are applied to the input latent and timestep embedding—this gives a scalar sensitivity score. Second, during inference, these sensitivity scores guide a caching policy: timesteps with low sensitivity are flagged as candidates for reusing cached outputs from previous timesteps, while high-sensitivity timesteps must be computed fresh to maintain quality. The framework then determines the optimal set of timesteps to actually compute versus reuse, minimizing total forward passes while keeping accumulated error below a quality threshold. The caching logic can be implemented as a simple scheduler that decides at each step whether to execute the model or retrieve a cached result.
Production Impact
For teams deploying diffusion-based video or image generation services, this technique offers direct latency and compute cost reduction without retraining. A typical diffusion model requires 50-100+ denoising steps; if sensitivity analysis shows 30-40% of timesteps can safely reuse cached values, you cut inference time by roughly that percentage with minimal quality loss. The training-free nature is critical for production—you can apply this to existing deployed models without the cost and complexity of fine-tuning. The main trade-off is upfront sensitivity analysis overhead (computing perturbation responses for all timesteps), but this is a one-time cost per model. You'd need to validate that sensitivity scores remain stable across different input domains (e.g., different video types) before deploying widely, and you'd likely implement this as a configurable knob (quality vs. speed) that users can tune for their latency budget.
Limitations and When Not to Use This
The paper assumes that sensitivity computed under small perturbations accurately predicts caching error under larger reuse distances—this may break down if you try to reuse values from timesteps far apart. The approach requires computing sensitivity scores for a given model architecture, meaning the analysis must be redone if the model changes, which could be problematic in rapidly evolving model landscapes. The framework is demonstrated on diffusion models but generalization to other sequential generation paradigms (autoregressive transformers, flow matching) is unclear. Additionally, the paper doesn't address how sensitivity-aware caching interacts with other acceleration techniques like timestep skipping, early stopping, or quantization—combining multiple techniques may introduce unexpected quality cliffs that the sensitivity metric doesn't capture.
Research Context
This work builds on the emerging line of research in training-free diffusion acceleration, following prior work on timestep skipping, output reuse, and cached attention mechanisms. It's motivated by the high computational cost of diffusion models in video generation (a key application area as of 2026) where inference speed directly impacts practical usability. The sensitivity-based perspective connects to broader work in neural network robustness and input-output stability analysis. The paper likely contributes to the standardization of caching in diffusion inference—moving from ad-hoc heuristics toward principled, theoretically-grounded methods that can be compared and composed with other acceleration techniques.
:::tip Subscribe Get weekly breakdowns of papers like this in AI Letters - the newsletter for engineers building production AI systems. :::