Diffusion Once and Done: Degradation-Aware LoRA for Efficient All-in-One Image Restoration
This addresses the need for efficient and adaptable image restoration methods for applications requiring diverse degradation handling, but it is incremental as it builds on existing diffusion models with novel fine-tuning techniques.
The paper tackles the problem of high inference costs and limited adaptability in all-in-one image restoration by proposing Diffusion Once and Done, which achieves superior restoration performance with only one-step sampling of Stable Diffusion models, outperforming existing approaches in visual quality and inference efficiency.
Diffusion models have revealed powerful potential in all-in-one image restoration (AiOIR), which is talented in generating abundant texture details. The existing AiOIR methods either retrain a diffusion model or fine-tune the pretrained diffusion model with extra conditional guidance. However, they often suffer from high inference costs and limited adaptability to diverse degradation types. In this paper, we propose an efficient AiOIR method, Diffusion Once and Done (DOD), which aims to achieve superior restoration performance with only one-step sampling of Stable Diffusion (SD) models. Specifically, multi-degradation feature modulation is first introduced to capture different degradation prompts with a pretrained diffusion model. Then, parameter-efficient conditional low-rank adaptation integrates the prompts to enable the fine-tuning of the SD model for adapting to different degradation types. Besides, a high-fidelity detail enhancement module is integrated into the decoder of SD to improve structural and textural details. Experiments demonstrate that our method outperforms existing diffusion-based restoration approaches in both visual quality and inference efficiency.