Boundary IoU: Improving Object-Centric Image Segmentation Evaluation
This work addresses the need for better segmentation evaluation metrics for researchers and practitioners, though it is incremental as it builds on existing IoU-based measures.
The authors tackled the problem of evaluating object-centric image segmentation by introducing Boundary IoU, a new measure that is more sensitive to boundary errors for large objects and less penalizing for small ones, with experiments showing it tracks boundary quality improvements overlooked by standard metrics.
We present Boundary IoU (Intersection-over-Union), a new segmentation evaluation measure focused on boundary quality. We perform an extensive analysis across different error types and object sizes and show that Boundary IoU is significantly more sensitive than the standard Mask IoU measure to boundary errors for large objects and does not over-penalize errors on smaller objects. The new quality measure displays several desirable characteristics like symmetry w.r.t. prediction/ground truth pairs and balanced responsiveness across scales, which makes it more suitable for segmentation evaluation than other boundary-focused measures like Trimap IoU and F-measure. Based on Boundary IoU, we update the standard evaluation protocols for instance and panoptic segmentation tasks by proposing the Boundary AP (Average Precision) and Boundary PQ (Panoptic Quality) metrics, respectively. Our experiments show that the new evaluation metrics track boundary quality improvements that are generally overlooked by current Mask IoU-based evaluation metrics. We hope that the adoption of the new boundary-sensitive evaluation metrics will lead to rapid progress in segmentation methods that improve boundary quality.