Prompt What You Need: Enhancing Segmentation in Rainy Scenes with Anchor-based Prompting
This work addresses the problem of accurate semantic segmentation in adverse weather conditions for applications like autonomous driving, but it is incremental as it builds on existing models and methods.
The paper tackles semantic segmentation in rainy scenes by proposing a framework that combines semi-supervised learning with a pre-trained foundation model, using anchor-based prompting and mask filtering to improve performance, achieving first prize in a challenge with superior results on the Rainy WCity dataset.
Semantic segmentation in rainy scenes is a challenging task due to the complex environment, class distribution imbalance, and limited annotated data. To address these challenges, we propose a novel framework that utilizes semi-supervised learning and pre-trained segmentation foundation model to achieve superior performance. Specifically, our framework leverages the semi-supervised model as the basis for generating raw semantic segmentation results, while also serving as a guiding force to prompt pre-trained foundation model to compensate for knowledge gaps with entropy-based anchors. In addition, to minimize the impact of irrelevant segmentation masks generated by the pre-trained foundation model, we also propose a mask filtering and fusion mechanism that optimizes raw semantic segmentation results based on the principle of minimum risk. The proposed framework achieves superior segmentation performance on the Rainy WCity dataset and is awarded the first prize in the sub-track of STRAIN in ICME 2023 Grand Challenges.