Differences in Detection: Explainability Where it Matters
For researchers and practitioners in object detection, DnD provides a more direct and intuitive way to compare models and identify specific error patterns, though it is an incremental improvement over existing comparison methods.
The paper proposes Differences in Detection (DnD), a method to compare two object detection models by directly analyzing their shared and unique detections, enabling more intuitive error analysis than standard metrics like mAP. The method can guide explainability techniques toward metric-relevant examples.
We propose Differences in Detection (DnD), an intuitive method to compare two object detection models. Based on the same matching algorithm, it complements the standard metrics of mean Average Precision ($mAP$) and TIDE error analysis with the ability to compare two models directly. More specifically, we calculate the intersection of ground truth labels that are recognized by both models, followed by the corresponding difference sets and the complement set of ground truth labels that are missed by both models. The resulting comparison is more direct and intuitive than a comparison of independent summary statistics. It reveals individual and shared mistakes and becomes particularly interesting when combined with error types. In this case, the differences in detection errors can be analyzed naturally in a standard confusion matrix. While valuable in itself, we believe that one of the best applications of DnD is to guide explainability methods such as ODAM towards metric-relevant examples, grounded in structured subsets. The code for our method is available here: https://github.com/JohannesTheo/differences-in-detection