Improving Image-to-Image Translation via a Rectified Flow Reformulation
| Authors | Satoshi Iizuka et al. |
| Year | 2026 |
| Field | Computer Vision |
| arXiv | 2603.20186 |
| Download | |
| Categories | cs.CV |
Abstract
In this work, we propose Image-to-Image Rectified Flow Reformulation (I2I-RFR), a practical plug-in reformulation that recasts standard I2I regression networks as continuous-time transport models. While pixel-wise I2I regression is simple, stable, and easy to adapt across tasks, it often over-smooths ill-posed and multimodal targets, whereas generative alternatives often require additional components, task-specific tuning, and more complex training and inference pipelines. Our method augments the backbone input by channel-wise concatenation with a noise-corrupted version of the ground-truth target and optimizes a simple t-reweighted pixel loss. This objective admits a rectified-flow interpretation via an induced velocity field, enabling ODE-based progressive refinement at inference time while largely preserving the standard supervised training pipeline. In most cases, adopting I2I-RFR requires only expanding the input channels, and inference can be performed with a few explicit solver steps (e.g., 3 steps) without distillation. Extensive experiments across multiple image-to-image translation and video restoration tasks show that I2I-RFR generally improves performance across a wide range of tasks and backbones, with particularly clear gains in perceptual quality and detail preservation. Overall, I2I-RFR provides a lightweight way to incorporate continuous-time refinement into conventional I2I models without requiring a heavy generative pipeline.
Engineering Breakdown
Plain English
This paper presents I2I-RFR, a method that transforms standard image-to-image regression networks into continuous-time transport models using rectified flow theory. The core problem is that pixel-wise regression networks produce over-smoothed outputs for ill-posed and multimodal tasks, while generative alternatives are complex and require careful tuning. The authors' solution augments the input by concatenating noise-corrupted versions of the target image and applies a t-reweighted pixel loss, which mathematically corresponds to an induced velocity field in rectified flow. This enables progressive refinement during inference using ODE solvers, offering a middle ground between simple regression and full generative modeling.
Core Technical Contribution
The key novelty is recasting I2I regression as a rectified flow problem through a simple input augmentation and loss reweighting scheme, without requiring architectural changes or complex training procedures. Unlike prior generative I2I methods that need adversarial losses, diffusion score networks, or other specialized components, this approach keeps the backbone network unchanged and adds only channel-wise concatenation with corrupted targets and a velocity field objective. The insight is that by explicitly modeling the transport trajectory from noise to the target, the network learns to produce sharper, more diverse outputs than standard L2 regression, while remaining more stable than VAE or GAN-based alternatives. This plug-in formulation admits a continuous-time interpretation that enables variable-cost inference via ODE-based progressive refinement.
How It Works
The method takes a standard I2I regression network and modifies only its input and loss function. At training time, for each ground-truth target image, the network is given two inputs: the source image and a noise-corrupted version of the target image (sampled at some timestep t along the corruption path), concatenated along the channel dimension. The network outputs a prediction, and the loss is a t-reweighted pixel-wise MSE that emphasizes different timesteps differently—this weighting is derived from rectified flow theory and encourages the network to learn a velocity field that smoothly transports noise toward the clean target. At inference time, instead of single-pass regression, the method starts with pure noise and uses an ODE solver to progressively refine it by following the learned velocity field, similar to diffusion-based refinement but with the efficiency and stability of the regression backbone. The ODE solver can take varying numbers of steps, allowing a trade-off between output quality and computational cost without retraining.
Production Impact
For teams building image translation systems (super-resolution, style transfer, colorization, etc.), this approach offers a practical way to improve output sharpness and diversity without redesigning the entire pipeline. The plug-in nature means you can retrofit existing regression-based I2I models—no need to replace loss functions like adversarial or diffusion objectives, and inference remains GPU-friendly since it reuses the same backbone architecture trained on standard hardware. The progressive refinement at inference time is a significant advantage: you can trade latency for quality by choosing the number of ODE steps, which is valuable in real-time applications where sometimes a single fast forward pass is acceptable and sometimes you can afford multiple steps. However, the training cost increases because you must sample multiple noise levels (different t values) for each target, effectively increasing the per-batch compute overhead compared to standard regression; the method also assumes the corruption process (e.g., Gaussian noise) is well-defined and matches the test distribution.
Limitations and When Not to Use This
The paper's abstract does not specify performance numbers or provide empirical validation, making it difficult to assess whether the theoretical improvements translate to practical gains on standard benchmarks (LPIPS, FID, task-specific metrics). The method depends critically on the choice of noise corruption schedule and the t-reweighting scheme; if these are misaligned with the actual task, the velocity field may not learn meaningful refinement patterns, and outputs could degrade to the quality of standard regression. The approach assumes a single well-defined target distribution; in highly multimodal scenarios where multiple valid outputs exist (e.g., image completion), the rectified flow formulation may still converge to a single mode rather than enabling diverse sampling without additional modifications. Scaling to very high-resolution images (e.g., 4K) with ODE-based refinement could become computationally expensive at inference time if many steps are required, partially undermining the efficiency advantage over diffusion models.
Research Context
This work builds on the recent surge in rectified flow methods, which offer a simpler alternative to diffusion models by directly modeling transport between two distributions without explicit score estimation. The paper positions itself within the image-to-image translation literature, which has historically oscillated between simple pixel-regression baselines (fast, stable, but blurry) and generative models (sharper, more diverse, but complex and slow). The contribution bridges this gap by borrowing the mathematical framework of optimal transport and continuous normalizing flows, showing that rectified flow principles can be applied to conditional image generation. This work likely opens a research direction toward simpler, more modular generative models that can be plugged into existing regression architectures, potentially influencing how future I2I systems balance expressiveness and engineering simplicity.
:::tip Subscribe Get weekly breakdowns of papers like this in AI Letters - the newsletter for engineers building production AI systems. :::