UIR-LoRA: Achieving Universal Image Restoration through Multiple Low-Rank Adaptation
This work addresses image restoration for mixed degradations, offering an incremental improvement over existing unified methods by better utilizing multi-task learning.
The paper tackles the problem of multi-degradation image restoration by proposing a universal framework using multiple low-rank adapters (LoRA) to leverage commonalities and specificities across tasks, achieving higher fidelity and perceptual quality with better generalization than other unified models.
Existing unified methods typically treat multi-degradation image restoration as a multi-task learning problem. Despite performing effectively compared to single degradation restoration methods, they overlook the utilization of commonalities and specificities within multi-task restoration, thereby impeding the model's performance. Inspired by the success of deep generative models and fine-tuning techniques, we proposed a universal image restoration framework based on multiple low-rank adapters (LoRA) from multi-domain transfer learning. Our framework leverages the pre-trained generative model as the shared component for multi-degradation restoration and transfers it to specific degradation image restoration tasks using low-rank adaptation. Additionally, we introduce a LoRA composing strategy based on the degradation similarity, which adaptively combines trained LoRAs and enables our model to be applicable for mixed degradation restoration. Extensive experiments on multiple and mixed degradations demonstrate that the proposed universal image restoration method not only achieves higher fidelity and perceptual image quality but also has better generalization ability than other unified image restoration models. Our code is available at https://github.com/Justones/UIR-LoRA.