Helios: Real Real-Time Long Video Generation Model
| Authors | Shenghai Yuan et al. |
| Year | 2026 |
| Field | Computer Vision |
| arXiv | 2603.04379 |
| Download | |
| Categories | cs.CV |
Abstract
We introduce Helios, the first 14B video generation model that runs at 19.5 FPS on a single NVIDIA H100 GPU and supports minute-scale generation while matching the quality of a strong baseline. We make breakthroughs along three key dimensions: (1) robustness to long-video drifting without commonly used anti-drifting heuristics such as self-forcing, error-banks, or keyframe sampling; (2) real-time generation without standard acceleration techniques such as KV-cache, sparse/linear attention, or quantization; and (3) training without parallelism or sharding frameworks, enabling image-diffusion-scale batch sizes while fitting up to four 14B models within 80 GB of GPU memory. Specifically, Helios is a 14B autoregressive diffusion model with a unified input representation that natively supports T2V, I2V, and V2V tasks. To mitigate drifting in long-video generation, we characterize typical failure modes and propose simple yet effective training strategies that explicitly simulate drifting during training, while eliminating repetitive motion at its source. For efficiency, we heavily compress the historical and noisy context and reduce the number of sampling steps, yielding computational costs comparable to -- or lower than -- those of 1.3B video generative models. Moreover, we introduce infrastructure-level optimizations that accelerate both inference and training while reducing memory consumption. Extensive experiments demonstrate that Helios consistently outperforms prior methods on both short- and long-video generation. We plan to release the code, base model, and distilled model to support further development by the community.
Engineering Breakdown
Plain English
Helios is a 14-billion-parameter video generation model that generates minute-long videos in real time at 19.5 frames per second on a single H100 GPU while matching the quality of existing baselines. The paper tackles three major practical challenges: preventing quality degradation over long sequences (drift), achieving real-time speed without typical acceleration tricks like KV-caching or quantization, and fitting multiple large models in 80GB of GPU memory through an efficient training approach that doesn't require distributed parallelism. The key insight is using a unified autoregressive diffusion architecture that handles video generation natively without the engineering complexity of prior systems.
Core Technical Contribution
The core technical novelty is demonstrating that a large autoregressive diffusion model can generate coherent long-form video without specialized anti-drifting techniques (no self-forcing, error-banks, or keyframe sampling strategies) that prior work relied on. The authors achieve real-time throughput through careful architectural design and memory efficiency rather than algorithmic tricks—no sparse attention, linear attention, or post-training quantization needed. A third major contribution is showing that video models can be trained without sharding frameworks or data parallelism, using gradient checkpointing and memory-efficient kernels to fit four 14B models in a single 80GB GPU, enabling batch sizes comparable to image diffusion model training.
How It Works
Helios uses a unified autoregressive diffusion framework where video frames are generated sequentially, one token or frame at a time, conditioned on all previously generated content. The input is a video prompt (text or image) that gets encoded into a latent representation, which the 14B transformer processes autoregressively—at each step, the model predicts the next diffusion noise level or latent tokens for the upcoming frame. The model employs a single unified representation space that avoids separate branching logic for different modalities, reducing implementation complexity and improving inference efficiency. During generation, the diffusion process iteratively refines frames through reverse diffusion steps, with the autoregressive structure ensuring temporal consistency by conditioning future predictions on concrete past outputs rather than hidden states. Training uses gradient checkpointing and activation recomputation to reduce peak memory, allowing reasonable batch sizes (8-16 typically) without distributed training, which is critical for stability and reproducibility.
Production Impact
For teams building video generation systems, Helios removes a major operational barrier: you can now run minute-scale video generation on a single H100 without renting multiple GPUs or using quantization that degrades output quality. The real-time performance (19.5 FPS) means streaming use cases—live video editing, interactive content generation, real-time style transfer—become feasible on standard infrastructure. The simplified training procedure (no distributed sharding needed) significantly lowers the barrier to fine-tuning or adapting the model; practitioners can train on a single GPU with modest batch sizes and expect stable convergence, unlike distributed approaches prone to synchronization bugs. However, trade-offs exist: the autoregressive generation approach is inherently sequential, so you cannot easily parallelize frame generation across multiple GPUs the way you can with diffusion models that generate all frames in parallel; if you have 8 GPUs, you cannot proportionally speed up generation by distributing it, which matters for batch inference scenarios.
Limitations and When Not to Use This
The paper does not address how to handle dynamic or interactive video where the user modifies or injects content mid-generation—the autoregressive design makes this harder than parallel approaches. The minute-scale generation claim is empirically validated but the absolute limits are unclear: can the model handle 10 minutes without degradation? Does quality degrade smoothly or hit a cliff at some length? The training procedure assumes access to high-quality, long-form video data at scale; the paper does not discuss data acquisition, filtering, or curation strategies, which are often the bottleneck in practice. Additionally, there is no analysis of failure modes in detail (e.g., when does drift still occur, what triggers it, how do you detect it), and generalization to out-of-distribution prompts or extreme aspect ratios is not explored.
Research Context
This work builds on the recent success of diffusion models and autoregressive transformers in vision, extending proven ideas from image generation (latent diffusion) and language (GPT-style autoregressive generation) to long-form video. It directly responds to prior video generation methods that either sacrificed quality for speed or required expensive multi-GPU setups and inference tricks; papers like Make-A-Video and Runway's GEN-3 pushed capability but left efficiency and drift as unsolved problems. The paper opens a research direction toward 'single-GPU, real-time capable' foundation models that can be fine-tuned and deployed without cloud infrastructure. It also challenges assumptions in the field—the result that you do not need anti-drifting heuristics suggests the root cause of prior drift was architectural or training-procedure specific, not fundamental to long-horizon generation.
:::tip Subscribe Get weekly breakdowns of papers like this in AI Letters - the newsletter for engineers building production AI systems. :::