Open-Vocabulary Object Detection with Meta Prompt Representation and Instance Contrastive Optimization
This addresses the limitation of object detectors in recognizing novel classes without extra data or complex techniques, offering a more efficient solution for applications requiring flexible object detection.
The paper tackles the problem of open-vocabulary object detection, where existing models overfit on base classes and rely on complex training, by proposing a framework with meta prompt representation and instance contrastive optimization, achieving improvements of +4.3% and +1.9% AP on COCO and Objects365 datasets compared to previous state-of-the-art methods.
Classical object detectors are incapable of detecting novel class objects that are not encountered before. Regarding this issue, Open-Vocabulary Object Detection (OVOD) is proposed, which aims to detect the objects in the candidate class list. However, current OVOD models are suffering from overfitting on the base classes, heavily relying on the large-scale extra data, and complex training process. To overcome these issues, we propose a novel framework with Meta prompt and Instance Contrastive learning (MIC) schemes. Firstly, we simulate a novel-class-emerging scenario to help the prompt learner that learns class and background prompts generalize to novel classes. Secondly, we design an instance-level contrastive strategy to promote intra-class compactness and inter-class separation, which benefits generalization of the detector to novel class objects. Without using knowledge distillation, ensemble model or extra training data during detector training, our proposed MIC outperforms previous SOTA methods trained with these complex techniques on LVIS. Most importantly, MIC shows great generalization ability on novel classes, e.g., with $+4.3\%$ and $+1.9\% \ \mathrm{AP}$ improvement compared with previous SOTA on COCO and Objects365, respectively.