Enhanced-alignment Measure for Binary Foreground Map Evaluation
This work addresses the need for better evaluation metrics in computer vision for tasks like segmentation, though it is incremental as it builds on existing measures.
The paper tackles the problem of evaluating binary foreground maps by proposing a novel E-measure that combines local pixel values with image-level mean to capture both global and local information, resulting in improvements of 9.08% to 19.65% in application ranking compared to existing measures.
The existing binary foreground map (FM) measures to address various types of errors in either pixel-wise or structural ways. These measures consider pixel-level match or image-level information independently, while cognitive vision studies have shown that human vision is highly sensitive to both global information and local details in scenes. In this paper, we take a detailed look at current binary FM evaluation measures and propose a novel and effective E-measure (Enhanced-alignment measure). Our measure combines local pixel values with the image-level mean value in one term, jointly capturing image-level statistics and local pixel matching information. We demonstrate the superiority of our measure over the available measures on 4 popular datasets via 5 meta-measures, including ranking models for applications, demoting generic, random Gaussian noise maps, ground-truth switch, as well as human judgments. We find large improvements in almost all the meta-measures. For instance, in terms of application ranking, we observe improvementrangingfrom9.08% to 19.65% compared with other popular measures.