SCoRe: Clean Image Generation from Diffusion Models Trained on Noisy Images
This addresses image quality degradation for users of diffusion models trained on noisy data, but it is incremental as it builds on existing spectral bias and SDEdit techniques.
The paper tackles the problem of diffusion models trained on noisy datasets reproducing high-frequency artifacts, proposing SCoRe, a training-free spectral regeneration method that suppresses corrupted high-frequency components and regenerates them via SDEdit, achieving substantial improvements over baselines on synthetic and real-world datasets.
Diffusion models trained on noisy datasets often reproduce high-frequency training artifacts, significantly degrading generation quality. To address this, we propose SCoRe (Spectral Cutoff Regeneration), a training-free, generation-time spectral regeneration method for clean image generation from diffusion models trained on noisy images. Leveraging the spectral bias of diffusion models, which infer high-frequency details from low-frequency cues, SCoRe suppresses corrupted high-frequency components of a generated image via a frequency cutoff and regenerates them via SDEdit. Crucially, we derive a theoretical mapping between the cutoff frequency and the SDEdit initialization timestep based on Radially Averaged Power Spectral Density (RAPSD), which prevents excessive noise injection during regeneration. Experiments on synthetic (CIFAR-10) and real-world (SIDD) noisy datasets demonstrate that SCoRe substantially outperforms post-processing and noise-robust baselines, restoring samples closer to clean image distributions without any retraining or fine-tuning.