LP-OVOD: Open-Vocabulary Object Detection by Linear Probing
This addresses a critical bottleneck in object detection for applications requiring generalization to new classes, though it is an incremental improvement over existing methods.
The paper tackles the problem of open-vocabulary object detection, where detectors must identify both seen and unseen classes without labeled examples, by proposing LP-OVOD to filter low-quality box proposals using a linear classifier on pseudo labels, achieving 40.5 AP_novel on COCO with ResNet50.
This paper addresses the challenging problem of open-vocabulary object detection (OVOD) where an object detector must identify both seen and unseen classes in test images without labeled examples of the unseen classes in training. A typical approach for OVOD is to use joint text-image embeddings of CLIP to assign box proposals to their closest text label. However, this method has a critical issue: many low-quality boxes, such as over- and under-covered-object boxes, have the same similarity score as high-quality boxes since CLIP is not trained on exact object location information. To address this issue, we propose a novel method, LP-OVOD, that discards low-quality boxes by training a sigmoid linear classifier on pseudo labels retrieved from the top relevant region proposals to the novel text. Experimental results on COCO affirm the superior performance of our approach over the state of the art, achieving $\textbf{40.5}$ in $\text{AP}_{novel}$ using ResNet50 as the backbone and without external datasets or knowing novel classes during training. Our code will be available at https://github.com/VinAIResearch/LP-OVOD.