Warp-as-History: Generalizable Camera-Controlled Video Generation from One Training Video
:::info Stub — Full Engineering Breakdown Coming This paper has a linked code implementation and was featured on Hugging Face Papers with 37 upvotes. A full breakdown with production viability rating, implementation notes, and honest limitations is being written. Subscribe to AI Letters → :::
| Authors | Yifan Wang & Tong He |
| Year | 2026 |
| HF Upvotes | 37 |
| arXiv | 2605.15182 |
| Download | |
| Code | https://github.com/yyfz/Warp-as-History |
Abstract
Camera-controlled video generation has made substantial progress, enabling generated videos to follow prescribed viewpoint trajectories. However, existing methods usually learn camera-specific conditioning through camera encoders, control branches, or attention and positional-encoding modifications, which often require post-training on large-scale camera-annotated videos. Training-free alternatives avoid such post-training, but often shift the cost to test-time optimization or extra denoising-time guidance. We propose Warp-as-History, a simple interface that turns camera-induced warps into camera-warped pseudo-history with target-frame positional alignment and visible-token selection. Given a target camera trajectory, we construct camera-warped pseudo-history from past observations and feed it through the model's visual-history pathway. Crucially, we align its positional encoding with the target frames being denoised and remove warped-history tokens without valid source observations. Without any training, architectural modification, or test-time optimization, this interface reveals a non-trivial zero-shot capability of a frozen video generation model to follow camera trajectories. Moreover, lightweight offline LoRA finetuning on only one camera-annotated video further improves this capability and generalizes to unseen videos, improving camera adherence, visual quality, and motion dynamics without test-time optimization or target-video adaptation. Extensive experiments on diverse datasets confirm the effectiveness of our method.
Engineering Breakdown
Plain English
This paper introduces Warp-as-History, a method for generating videos that follow camera trajectories without requiring expensive post-training on annotated video datasets. Instead of learning camera control through specialized encoders or attention mechanisms, the approach converts camera movements into warped pseudo-history frames that guide the generation process, achieving generalization from just a single training video.
Key Engineering Insight
By representing camera control as geometric warping of historical frames rather than learned conditioning, the method sidesteps the need for camera-specific model parameters entirely—making it a pure inference-time interface that works with existing pretrained diffusion models without architectural changes.
Why It Matters for Engineers
This matters because most production video generation systems either require expensive retraining on camera-annotated datasets (slow iteration cycle) or push computation to test-time optimization (slow inference). A training-free approach that works from single videos reduces both data collection overhead and deployment friction, making camera-controlled generation more practical for real products.
Research Context
Prior work split between two costly tradeoffs: fine-tuning approaches that need large annotated video collections, or test-time optimization that slows generation. Warp-as-History closes this gap by reframing camera control as a geometric problem rather than a learned feature, enabling plug-and-play camera control on any diffusion model and lowering the barrier to deploying controlled video generation in production systems.
:::tip Subscribe Get weekly breakdowns of papers like this in AI Letters - the newsletter for engineers building production AI systems. :::