Unified-modal Salient Object Detection via Adaptive Prompt Learning
This work addresses the practical deployment and computational cost issues for researchers and practitioners in computer vision by unifying single-modal and multi-modal salient object detection, though it is incremental as it builds on existing SOD methods.
The paper tackles the problem of labor-intensive and costly separate models for single-modal and multi-modal salient object detection by proposing UniSOD, a unified framework that uses adaptive prompt learning to handle both tasks, achieving overall performance improvement on 14 benchmark datasets for RGB, RGB-D, and RGB-T SOD.
Existing single-modal and multi-modal salient object detection (SOD) methods focus on designing specific architectures tailored for their respective tasks. However, developing completely different models for different tasks leads to labor and time consumption, as well as high computational and practical deployment costs. In this paper, we attempt to address both single-modal and multi-modal SOD in a unified framework called UniSOD, which fully exploits the overlapping prior knowledge between different tasks. Nevertheless, assigning appropriate strategies to modality variable inputs is challenging. To this end, UniSOD learns modality-aware prompts with task-specific hints through adaptive prompt learning, which are plugged into the proposed pre-trained baseline SOD model to handle corresponding tasks, while only requiring few learnable parameters compared to training the entire model. Each modality-aware prompt is generated from a switchable prompt generation block, which adaptively performs structural switching based on single-modal and multi-modal inputs without human intervention. Through end-to-end joint training, UniSOD achieves overall performance improvement on 14 benchmark datasets for RGB, RGB-D, and RGB-T SOD, which demonstrates that our method effectively and efficiently unifies single-modal and multi-modal SOD tasks.The code and results are available at https://github.com/Angknpng/UniSOD.