USD: Unknown Sensitive Detector Empowered by Decoupled Objectness and Segment Anything Model
This work addresses the challenge of detecting unknown objects in computer vision, which is crucial for real-world applications like autonomous driving, but it appears incremental as it builds on existing methods with specific enhancements.
The paper tackles the problem of detecting unknown objects in open-world object detection by addressing conflicts in learning objectness and classification boundaries, and leveraging the Segment Anything Model to enhance detection despite noisy outputs. The proposed Unknown Sensitive Detector achieves significant improvements, such as 14.3% to 29.1% higher Unknown Recall on benchmarks like M-OWODB and S-OWODB.
Open World Object Detection (OWOD) is a novel and challenging computer vision task that enables object detection with the ability to detect unknown objects. Existing methods typically estimate the object likelihood with an additional objectness branch, but ignore the conflict in learning objectness and classification boundaries, which oppose each other on the semantic manifold and training objective. To address this issue, we propose a simple yet effective learning strategy, namely Decoupled Objectness Learning (DOL), which divides the learning of these two boundaries into suitable decoder layers. Moreover, detecting unknown objects comprehensively requires a large amount of annotations, but labeling all unknown objects is both difficult and expensive. Therefore, we propose to take advantage of the recent Large Vision Model (LVM), specifically the Segment Anything Model (SAM), to enhance the detection of unknown objects. Nevertheless, the output results of SAM contain noise, including backgrounds and fragments, so we introduce an Auxiliary Supervision Framework (ASF) that uses a pseudo-labeling and a soft-weighting strategies to alleviate the negative impact of noise. Extensive experiments on popular benchmarks, including Pascal VOC and MS COCO, demonstrate the effectiveness of our approach. Our proposed Unknown Sensitive Detector (USD) outperforms the recent state-of-the-art methods in terms of Unknown Recall, achieving significant improvements of 14.3\%, 15.5\%, and 8.9\% on the M-OWODB, and 27.1\%, 29.1\%, and 25.1\% on the S-OWODB.