SCoRe: Clean Image Generation from Diffusion Models Trained on Noisy Images
| Authors | Yuta Matsuzaki et al. |
| Year | 2026 |
| Field | Computer Vision |
| arXiv | 2604.09436 |
| Download | |
| Categories | cs.CV |
Abstract
Diffusion models trained on noisy datasets often reproduce high-frequency training artifacts, significantly degrading generation quality. To address this, we propose SCoRe (Spectral Cutoff Regeneration), a training-free, generation-time spectral regeneration method for clean image generation from diffusion models trained on noisy images. Leveraging the spectral bias of diffusion models, which infer high-frequency details from low-frequency cues, SCoRe suppresses corrupted high-frequency components of a generated image via a frequency cutoff and regenerates them via SDEdit. Crucially, we derive a theoretical mapping between the cutoff frequency and the SDEdit initialization timestep based on Radially Averaged Power Spectral Density (RAPSD), which prevents excessive noise injection during regeneration. Experiments on synthetic (CIFAR-10) and real-world (SIDD) noisy datasets demonstrate that SCoRe substantially outperforms post-processing and noise-robust baselines, restoring samples closer to clean image distributions without any retraining or fine-tuning.
Engineering Breakdown
Plain English
This paper tackles a critical problem in generative AI: diffusion models trained on noisy datasets tend to reproduce training artifacts and high-frequency noise in their generated images, degrading output quality. The authors propose SCoRe (Spectral Cutoff Regeneration), a training-free method applied during image generation that identifies and removes corrupted high-frequency components while intelligently regenerating clean ones using SDEdit. The key insight is that diffusion models naturally infer high-frequency details from low-frequency information due to spectral bias, so SCoRe leverages this by cutting off corrupted frequencies and using a theoretically-grounded mapping (based on Radially Averaged Power Spectral Density) to determine the optimal SDEdit timestep for re-injection, avoiding excess noise.
Core Technical Contribution
The core novelty is a training-free, generation-time spectral filtering approach that exploits the inherent spectral bias of diffusion models to clean noisy-trained generators without retraining. The authors derive a theoretical mapping between frequency cutoff parameters and SDEdit initialization timesteps using RAPSD analysis, which is non-trivial because naive frequency suppression + re-injection risks introducing new noise artifacts. This differs from prior denoising approaches that either require retraining or lack principled frequency-to-timestep calibration, making SCoRe both computationally efficient (no training required) and theoretically grounded. The method is general-purpose—applicable to any diffusion model trained on noisy data regardless of noise type or corruption pattern.
How It Works
SCoRe operates in three sequential stages during generation. First, a diffusion model (trained on noisy data) generates an image normally via its full denoising trajectory. Second, a spectral analysis step examines the generated image's frequency content and identifies a cutoff frequency threshold above which high-frequency components are likely corrupted training artifacts; this cutoff is determined by analyzing the Radially Averaged Power Spectral Density to distinguish signal from noise. Third, the method applies a frequency-domain mask to suppress components above this cutoff, then uses SDEdit (a controlled diffusion editing technique) initialized at a carefully chosen timestep to regenerate those suppressed frequencies from the remaining clean low-frequency base. The theoretical contribution is a closed-form or empirically-validated mapping from the spectral cutoff frequency to the SDEdit timestep, preventing the common failure mode of injecting excessive noise during regeneration.
Production Impact
For teams deploying diffusion models on real-world noisy datasets (medical imaging, surveillance footage, user-generated content, etc.), SCoRe offers a zero-cost mitigation strategy—no retraining, no fine-tuning, applied purely at inference time. In a production pipeline, this means you can take an existing noisy-trained generator and apply SCoRe as a post-processing step with minimal integration overhead: add a spectral analysis module and an SDEdit subroutine to your generation code. The latency trade-off is moderate (SDEdit adds ~10-30% extra diffusion steps depending on the chosen timestep), but many applications can tolerate this cost in exchange for artifact-free outputs. The approach is particularly valuable for resource-constrained settings where retraining on cleaned data or gathering large clean datasets is infeasible, making it practical for edge deployment or rapid iteration scenarios.
Limitations and When Not to Use This
SCoRe assumes that training artifacts concentrate in high-frequency components and clean signal in low frequencies—this assumption may fail for certain corruption types (e.g., low-frequency bias, systematic dataset misalignment) or domains where semantically important details naturally live in high frequencies. The method requires careful tuning of the cutoff frequency threshold; if set too conservatively, artifacts remain; if too aggressively, legitimate details are erased and regeneration adds computational cost without benefit. The theoretical mapping between RAPSD cutoff and SDEdit timestep is derived for specific noise models and may not generalize perfectly to arbitrary corruption patterns not seen during analysis. Additionally, the paper does not address downstream performance (e.g., does artifact removal hurt task-specific metrics like classification accuracy on generated synthetic training data?), and the evaluation is likely limited to visual fidelity metrics rather than end-to-end application performance.
Research Context
This work builds on decades of spectral analysis in image processing and recent advances in diffusion model understanding, particularly the observation that diffusion models exhibit spectral bias (learning low-frequency structure before high-frequency details). It advances the emerging subfield of diffusion model robustness and post-hoc correction, alongside concurrent work on adapting diffusion models for noisy or imperfect training data. The paper contributes to the broader clean generation problem, which is critical as diffusion models scale to real-world datasets where perfect noise-free data is rare. This opens research directions in frequency-aware diffusion training (learning to predict which frequencies are corrupted) and adaptive spectral filtering (dynamically determining cutoff per-image based on intrinsic quality measures).
:::tip Subscribe Get weekly breakdowns of papers like this in AI Letters - the newsletter for engineers building production AI systems. :::