Open-Vocabulary One-Stage Detection with Hierarchical Visual-Language Knowledge Distillation
This addresses the challenge of detecting novel object categories beyond training data for computer vision applications, representing an incremental improvement by enhancing one-stage detectors to close the performance gap with two-stage methods.
The paper tackles the problem of open-vocabulary object detection in one-stage detectors, which suffer from performance degradation on unseen objects due to lack of class-agnostic proposals, and proposes a hierarchical knowledge distillation method that achieves significant gains, such as 11.9% and 6.7% AP50 improvements over previous best one-stage detectors in zero-shot and generalized zero-shot settings.
Open-vocabulary object detection aims to detect novel object categories beyond the training set. The advanced open-vocabulary two-stage detectors employ instance-level visual-to-visual knowledge distillation to align the visual space of the detector with the semantic space of the Pre-trained Visual-Language Model (PVLM). However, in the more efficient one-stage detector, the absence of class-agnostic object proposals hinders the knowledge distillation on unseen objects, leading to severe performance degradation. In this paper, we propose a hierarchical visual-language knowledge distillation method, i.e., HierKD, for open-vocabulary one-stage detection. Specifically, a global-level knowledge distillation is explored to transfer the knowledge of unseen categories from the PVLM to the detector. Moreover, we combine the proposed global-level knowledge distillation and the common instance-level knowledge distillation to learn the knowledge of seen and unseen categories simultaneously. Extensive experiments on MS-COCO show that our method significantly surpasses the previous best one-stage detector with 11.9\% and 6.7\% $AP_{50}$ gains under the zero-shot detection and generalized zero-shot detection settings, and reduces the $AP_{50}$ performance gap from 14\% to 7.3\% compared to the best two-stage detector.