Beyond Pixel Overlap: A Framework for Decomposing Segmentation Evaluation Metrics
For researchers in binary segmentation, this provides a structured way to understand and design evaluation metrics, but the contribution is primarily analytical rather than empirical.
The paper proposes a framework that decomposes segmentation evaluation metrics into five modular stages, enabling systematic analysis and design of task-aware evaluation protocols. The framework reveals how newer metrics address specific limitations of earlier ones.
Evaluation metrics are central to binary target segmentation because they determine how progress is measured, compared, and interpreted. In this paper, target denotes the task-defined positive region to be segmented rather than a generic foreground object. It may be salient, camouflaged, transparent, glass-like, mirror-like, shadow-like, lesion-like, or defined by other application-specific semantics. We treat existing metrics as compositions of modular design choices rather than isolated formulas. The proposed framework decomposes each metric into five stages covering prediction representation, target extraction, target matching, score computation, and metric reporting. We use this framework to analyze representative metrics and show how newer metrics address specific limits in earlier protocols. The stage choices keep each metric's assumptions visible. We then discuss the design space opened by the framework and its implications for task-aware evaluation protocols. Reference code is available at https://github.com/lartpang/PySODMetrics.