HDR Video Generation via Latent Alignment with Logarithmic Encoding
| Authors | Naomi Ken Korem et al. |
| Year | 2026 |
| HF Upvotes | 5 |
| arXiv | 2604.11788 |
| Download | |
| HF Page | View on Hugging Face |
Abstract
High dynamic range (HDR) imagery offers a rich and faithful representation of scene radiance, but remains challenging for generative models due to its mismatch with the bounded, perceptually compressed data on which these models are trained. A natural solution is to learn new representations for HDR, which introduces additional complexity and data requirements. In this work, we show that HDR generation can be achieved in a much simpler way by leveraging the strong visual priors already captured by pretrained generative models. We observe that a logarithmic encoding widely used in cinematic pipelines maps HDR imagery into a distribution that is naturally aligned with the latent space of these models, enabling direct adaptation via lightweight fine-tuning without retraining an encoder. To recover details that are not directly observable in the input, we further introduce a training strategy based on camera-mimicking degradations that encourages the model to infer missing high dynamic range content from its learned priors. Combining these insights, we demonstrate high-quality HDR video generation using a pretrained video model with minimal adaptation, achieving strong results across diverse scenes and challenging lighting conditions. Our results indicate that HDR, despite representing a fundamentally different image formation regime, can be handled effectively without redesigning generative models, provided that the representation is chosen to align with their learned priors.
Engineering Breakdown
Plain English
This paper solves the problem of generating high dynamic range (HDR) video using existing pretrained generative models, which were trained on standard dynamic range (SDR) data that doesn't naturally represent the full brightness and color range of HDR. The key insight is that logarithmic encoding—a standard technique from film production—transforms HDR values into a distribution that aligns well with the latent space of pretrained diffusion models, allowing HDR generation through simple fine-tuning rather than retraining from scratch. The authors show this approach works without requiring custom encoders or extensive retraining, making it practical and efficient. This bridges the gap between what generative models naturally understand and what HDR imagery requires, enabling high-quality HDR video synthesis with minimal additional complexity.
Core Technical Contribution
The core novelty is the discovery that logarithmic encoding creates a natural alignment between HDR image distributions and the latent spaces of pretrained generative models trained on SDR data. Rather than building new architectures or training procedures specifically for HDR, the authors show that the log-space representation of HDR data falls naturally within the manifold that existing models already understand. This enables lightweight fine-tuning—essentially adapting the model with minimal parameter updates—without requiring expensive encoder retraining or collecting massive HDR datasets. The insight is elegant: instead of forcing models to learn new representations, use a mathematical transformation (logarithmic encoding) that makes HDR data look like what the model already expects.
How It Works
The approach starts with standard SDR imagery and the corresponding HDR ground truth, both transformed into logarithmic space where pixel values are mapped via log(luminance) rather than linear intensity. The pretrained generative model (likely a diffusion model based on context) already has a learned latent space optimized for SDR-like distributions; the log-encoded HDR data aligns with this latent geometry because log compression mirrors the perceptual properties both SDR and the model's training implicitly capture. During inference, an SDR image (or conditioning signal) is encoded into the model's latent space, and the model generates latents for the corresponding HDR image through standard diffusion sampling or similar generation process. The generated latent is decoded back into linear HDR space, recovering the full dynamic range. The key efficiency gain is that the encoder and decoder remain frozen; only the generative model itself receives lightweight fine-tuning to learn the SDR-to-HDR mapping, reducing compute and data requirements dramatically compared to training from scratch.
Production Impact
For production video systems, this means HDR generation becomes practical without massive computational investment or proprietary HDR datasets. A team could take an existing pretrained diffusion model (widely available as open-source checkpoints) and fine-tune it on relatively modest HDR data to generate high-quality HDR video from SDR inputs—useful for up-conversion in streaming pipelines, content creation, and archival enhancement. The lightweight fine-tuning approach (likely LoRA-style parameter-efficient methods) reduces training time from weeks to days or hours, and inference latency remains near the original model's speed since the architecture doesn't change. Trade-offs include the need for paired SDR-HDR training data (not always abundant), and the log-encoding assumption may not hold perfectly for all edge cases like very bright or very dark scenes where perceptual properties differ. Integration is straightforward: plug the fine-tuned model into existing generation pipelines as a drop-in replacement for the base model, with log/antilog conversions as pre/post-processing steps.
Limitations and When Not to Use This
The approach assumes log-space alignment holds across all lighting conditions and color gamuts, but extreme highlights, shadows, or out-of-gamut colors may break this assumption—the paper likely doesn't address these edge cases thoroughly. The method requires paired SDR-HDR training data, which is expensive and limited in volume compared to SDR-only datasets; scaling to diverse content types (sports, nature, animation) may require prohibitive data collection. The paper doesn't discuss how this interacts with other HDR complexities like tone mapping reversal, color grading consistency, or temporal coherence in video (ghosting artifacts, flicker across frames). Finally, the approach is fundamentally constrained to the quality ceiling of the pretrained model; if the base model has limitations in fine detail, motion, or semantic understanding, fine-tuning won't overcome them—follow-up work likely needs to explore training HDR-aware base models from the ground up.
Research Context
This work builds on the success of large pretrained diffusion models for image and video generation, recognizing that these models capture rich visual priors even when trained on bounded SDR data. It follows the trend of leveraging pretrained models via fine-tuning and parameter-efficient adaptation (LoRA, adapters) rather than expensive retraining, similar to how CLIP-based approaches enable domain adaptation across tasks. The paper advances HDR research, which has historically struggled because most datasets and models are SDR-centric; prior work either built HDR-specific models from scratch or used tone-mapping heuristics that lose information. By showing that a simple mathematical transformation (log encoding) solves the distribution mismatch, this work opens a research direction: exploring what other domain or modality gaps can be bridged by finding the right representation space that aligns disparate datasets with pretrained models.
:::tip Subscribe Get weekly breakdowns of papers like this in AI Letters - the newsletter for engineers building production AI systems. :::