Training-Free Metrics for Synthetic Object Detection Data: A Proxy for Detector Performance
For researchers using synthetic data for object detection, this metric eliminates the need for expensive downstream training to evaluate data quality.
The authors propose CCDM, a family of pre-computable metrics that predict the utility of synthetic training data for object detection without training a downstream model. On VisDrone-DET, CCDM achieves a perfect Spearman correlation of 1.0 with YOLOv8 performance, outperforming existing metrics.
With the recent advent of image generative models, synthetic data are increasingly being used to supplement limited real datasets for training computer vision models. However, not all synthetic datasets improve performance equally, and their effectiveness can only be assessed by training a downstream model, which is computationally expensive and time-consuming. This problem is pronounced in the task of object detection, where the required annotations are much more dense due to bounding boxes. In this paper, we propose a pre-computable metric family, dubbed Conditional-Composition Domain Match (CCDM), which serves as a proxy for the relative utility of candidate synthetic training sets for downstream detection. Experiments on the VisDrone-DET dataset show that the CCDM metric families achieve a Spearman correlation of 1.0 with the downstream performance of YOLOv8, clearly outperforming existing metrics for synthetic image evaluation.