Forcing-KV: Hybrid KV Cache Compression for Efficient Autoregressive Video Diffusion Models
:::info Stub — Full Engineering Breakdown Coming This paper has a linked code implementation and was featured on Hugging Face Papers with 8 upvotes. A full breakdown with production viability rating, implementation notes, and honest limitations is being written. Subscribe to AI Letters → :::
| Authors | Yicheng Ji et al. |
| Year | 2026 |
| HF Upvotes | 8 |
| arXiv | 2605.09681 |
| Download | |
| Code | https://github.com/zju-jiyicheng/Forcing-KV |
Abstract
Autoregressive (AR) video diffusion models adopt a streaming generation framework, enabling long-horizon video generation with real-time responsiveness, as exemplified by the Self Forcing training paradigm. However, existing AR video diffusion models still suffer from significant attention complexity and severe memory overhead due to the redundant key-value (KV) caches across historical frames, which limits scalability. In this paper, we tackle this challenge by introducing KV cache compression into autoregressive video diffusion. We observe that attention heads in mainstream AR diffusion models exhibit markedly distinct attention patterns and functional roles that remain stable across samples and denoising steps. Building on our empirical study of head-wise functional specialization, we divide the attention heads into two categories: static heads, which focus on transitions across autoregressive chunks and intra-frame fidelity, and dynamic heads, which govern inter-frame motion and consistency. We then propose Forcing-KV, a hybrid KV cache compression strategy that performs structured static pruning for static heads and dynamic pruning based on segment-wise similarity for dynamic heads. While maintaining output quality, our method achieves a generation speed of over 29 frames per second on a single NVIDIA H200 GPU along with 30% cache memory reduction, delivering up to 1.35x and 1.50x speedups on LongLive and Self Forcing at 480P resolution, and further scaling to 2.82x speedup at 1080P resolution. Code and demo videos are provided at https://zju-jiyicheng.github.io/Forcing-KV-Page.
Engineering Breakdown
Plain English
This paper addresses memory overhead in autoregressive video diffusion models by compressing key-value (KV) caches across historical frames during generation. The key finding is that attention heads have distinct, stable functional roles across different samples and denoising steps, which enables selective compression of KV caches without significantly degrading output quality.
Key Engineering Insight
Attention heads in AR video diffusion models specialize in different functions (some focus on temporal consistency, others on spatial detail), and these roles remain consistent. This stability enables head-specific compression strategies instead of uniform quantization, potentially reducing memory overhead while preserving generation quality.
Why It Matters for Engineers
Video generation at scale is memory-bound—KV cache growth is linear with sequence length and becomes the bottleneck for longer sequences. This work directly addresses production scaling challenges: if compression works well, you can generate longer videos with smaller GPUs or batch more requests per GPU, directly improving throughput and reducing inference costs.
Research Context
Prior AR video diffusion work (like Self Forcing) enabled real-time streaming generation but created massive KV cache footprints. This paper advances the field by identifying that not all cached attention is equally important, enabling selective compression. This finding opens the door to hybrid compression strategies (some heads pruned, others quantized) that could make AR video generation practical for resource-constrained deployments.
:::tip Subscribe Get weekly breakdowns of papers like this in AI Letters - the newsletter for engineers building production AI systems. :::