Human Perception-based Evaluation Criterion for Ultra-high Resolution Cell Membrane Segmentation
This work addresses bottlenecks in biological and medical image analysis by providing a dataset and evaluation method for cell membrane segmentation, though it is incremental as it builds on existing segmentation methods.
The paper tackles the lack of high-quality data and suitable evaluation criteria in cell membrane segmentation by introducing U-RISC, the largest annotated Electron Microscopy dataset for cell membranes, and proposes a new evaluation criterion called Perceptual Hausdorff Distance (PHD) to align with human perception, validated through a subjective experiment with twenty people.
Computer vision technology is widely used in biological and medical data analysis and understanding. However, there are still two major bottlenecks in the field of cell membrane segmentation, which seriously hinder further research: lack of sufficient high-quality data and lack of suitable evaluation criteria. In order to solve these two problems, this paper first proposes an Ultra-high Resolution Image Segmentation dataset for the Cell membrane, called U-RISC, the largest annotated Electron Microscopy (EM) dataset for the Cell membrane with multiple iterative annotations and uncompressed high-resolution raw data. During the analysis process of the U-RISC, we found that the current popular segmentation evaluation criteria are inconsistent with human perception. This interesting phenomenon is confirmed by a subjective experiment involving twenty people. Furthermore, to resolve this inconsistency, we propose a new evaluation criterion called Perceptual Hausdorff Distance (PHD) to measure the quality of cell membrane segmentation results. Detailed performance comparison and discussion of classic segmentation methods along with two iterative manual annotation results under existing evaluation criteria and PHD is given.