Pitfalls of topology-aware image segmentation
This work addresses evaluation problems for researchers and practitioners in medical imaging, but it is incremental as it focuses on improving existing benchmarking practices rather than introducing new methods.
The paper identifies critical pitfalls in benchmarking topology-aware medical image segmentation methods, such as inadequate connectivity choices and flawed ground truth annotations, and shows these issues significantly affect method evaluation and ranking, proposing actionable recommendations for robust standards.
Topological correctness, i.e., the preservation of structural integrity and specific characteristics of shape, is a fundamental requirement for medical imaging tasks, such as neuron or vessel segmentation. Despite the recent surge in topology-aware methods addressing this challenge, their real-world applicability is hindered by flawed benchmarking practices. In this paper, we identify critical pitfalls in model evaluation that include inadequate connectivity choices, overlooked topological artifacts in ground truth annotations, and inappropriate use of evaluation metrics. Through detailed empirical analysis, we uncover these issues' profound impact on the evaluation and ranking of segmentation methods. Drawing from our findings, we propose a set of actionable recommendations to establish fair and robust evaluation standards for topology-aware medical image segmentation methods.