Vision-Language Gradient Descent-driven All-in-One Deep Unfolding Networks
This addresses the adaptability challenge in image restoration for scenarios with diverse sensor or environmental conditions, though it is incremental as it builds on existing deep unfolding networks.
The paper tackled the problem of dynamic image degradations like noise, blur, and lighting inconsistencies by proposing VLU-Net, a unified deep unfolding network that uses vision-language guidance to automatically handle multiple degradation types, achieving improvements of 3.74 dB on SOTS dehazing and 1.70 dB on Rain100L deraining datasets.
Dynamic image degradations, including noise, blur and lighting inconsistencies, pose significant challenges in image restoration, often due to sensor limitations or adverse environmental conditions. Existing Deep Unfolding Networks (DUNs) offer stable restoration performance but require manual selection of degradation matrices for each degradation type, limiting their adaptability across diverse scenarios. To address this issue, we propose the Vision-Language-guided Unfolding Network (VLU-Net), a unified DUN framework for handling multiple degradation types simultaneously. VLU-Net leverages a Vision-Language Model (VLM) refined on degraded image-text pairs to align image features with degradation descriptions, selecting the appropriate transform for target degradation. By integrating an automatic VLM-based gradient estimation strategy into the Proximal Gradient Descent (PGD) algorithm, VLU-Net effectively tackles complex multi-degradation restoration tasks while maintaining interpretability. Furthermore, we design a hierarchical feature unfolding structure to enhance VLU-Net framework, efficiently synthesizing degradation patterns across various levels. VLU-Net is the first all-in-one DUN framework and outperforms current leading one-by-one and all-in-one end-to-end methods by 3.74 dB on the SOTS dehazing dataset and 1.70 dB on the Rain100L deraining dataset.