Multiscale IoU: A Metric for Evaluation of Salient Object Detection with Fine Structures
This addresses the need for better evaluation methods in object detection, particularly for applications requiring fine structural accuracy, though it is incremental as it builds on existing metrics.
The paper tackles the problem of object-detection algorithms ignoring fine structures in detected objects by proposing Multiscale IoU (MIoU), a new metric combining IoU and fractal dimension to evaluate at multiple resolutions, showing it is sensitive to boundary details overlooked by existing metrics like IoU and f1-score.
General-purpose object-detection algorithms often dismiss the fine structure of detected objects. This can be traced back to how their proposed regions are evaluated. Our goal is to renegotiate the trade-off between the generality of these algorithms and their coarse detections. In this work, we present a new metric that is a marriage of a popular evaluation metric, namely Intersection over Union (IoU), and a geometrical concept, called fractal dimension. We propose Multiscale IoU (MIoU) which allows comparison between the detected and ground-truth regions at multiple resolution levels. Through several reproducible examples, we show that MIoU is indeed sensitive to the fine boundary structures which are completely overlooked by IoU and f1-score. We further examine the overall reliability of MIoU by comparing its distribution with that of IoU on synthetic and real-world datasets of objects. We intend this work to re-initiate exploration of new evaluation methods for object-detection algorithms.