Low-rank Adaptation-based All-Weather Removal for Autonomous Navigation
This addresses the inefficiency of retraining or fine-tuning for autonomous navigation systems in adverse weather, though it appears incremental as it builds on existing LoRA techniques.
The paper tackles the problem of all-weather image restoration models struggling with out-of-distribution weather conditions, which limits their real-world effectiveness for autonomous navigation, by proposing LoRA-Align (LoRA-A), a method that adapts pre-trained models to novel weather tasks while preserving original task knowledge, achieving improved performance on semantic segmentation and depth estimation.
All-weather image restoration (AWIR) is crucial for reliable autonomous navigation under adverse weather conditions. AWIR models are trained to address a specific set of weather conditions such as fog, rain, and snow. But this causes them to often struggle with out-of-distribution (OoD) samples or unseen degradations which limits their effectiveness for real-world autonomous navigation. To overcome this issue, existing models must either be retrained or fine-tuned, both of which are inefficient and impractical, with retraining needing access to large datasets, and fine-tuning involving many parameters. In this paper, we propose using Low-Rank Adaptation (LoRA) to efficiently adapt a pre-trained all-weather model to novel weather restoration tasks. Furthermore, we observe that LoRA lowers the performance of the adapted model on the pre-trained restoration tasks. To address this issue, we introduce a LoRA-based fine-tuning method called LoRA-Align (LoRA-A) which seeks to align the singular vectors of the fine-tuned and pre-trained weight matrices using Singular Value Decomposition (SVD). This alignment helps preserve the model's knowledge of its original tasks while adapting it to unseen tasks. We show that images restored with LoRA and LoRA-A can be effectively used for computer vision tasks in autonomous navigation, such as semantic segmentation and depth estimation.