From COCO to COCO-FP: A Deep Dive into Background False Positives for COCO Detectors
This addresses the critical issue of false alarms in real-world applications like fire and smoke detection, though it is incremental as it focuses on evaluation rather than a new detection method.
The study tackled the problem of background false positives in object detectors by introducing COCO-FP, a new evaluation dataset derived from ImageNet-1K, which revealed significant performance drops, such as YOLOv9-E's AP50 decreasing from 72.8 to 65.7.
Reducing false positives is essential for enhancing object detector performance, as reflected in the mean Average Precision (mAP) metric. Although object detectors have achieved notable improvements and high mAP scores on the COCO dataset, analysis reveals limited progress in addressing false positives caused by non-target visual clutter-background objects not included in the annotated categories. This issue is particularly critical in real-world applications, such as fire and smoke detection, where minimizing false alarms is crucial. In this study, we introduce COCO-FP, a new evaluation dataset derived from the ImageNet-1K dataset, designed to address this issue. By extending the original COCO validation dataset, COCO-FP specifically assesses object detectors' performance in mitigating background false positives. Our evaluation of both standard and advanced object detectors shows a significant number of false positives in both closed-set and open-set scenarios. For example, the AP50 metric for YOLOv9-E decreases from 72.8 to 65.7 when shifting from COCO to COCO-FP. The dataset is available at https://github.com/COCO-FP/COCO-FP.