The Weakly-Labeled Rand Index
This addresses the challenge of scoring segmentations in remote sensing imagery with uncertain boundaries, but it is incremental as it adapts an existing metric for a specific domain.
The paper tackles the problem of evaluating image segmentations in Synthetic Aperture Sonar (SAS) surveys, where weak labels are needed due to uncertain boundaries, by introducing a modified Rand index for weakly-labeled data. Results show that this new index scores segmentations appropriately in reference to qualitative performance and is more suitable than traditional metrics for weakly-labeled data.
Synthetic Aperture Sonar (SAS) surveys produce imagery with large regions of transition between seabed types. Due to these regions, it is difficult to label and segment the imagery and, furthermore, challenging to score the image segmentations appropriately. While there are many approaches to quantify performance in standard crisp segmentation schemes, drawing hard boundaries in remote sensing imagery where gradients and regions of uncertainty exist is inappropriate. These cases warrant weak labels and an associated appropriate scoring approach. In this paper, a labeling approach and associated modified version of the Rand index for weakly-labeled data is introduced to address these issues. Results are evaluated with the new index and compared to traditional segmentation evaluation methods. Experimental results on a SAS data set containing must-link and cannot-link labels show that our Weakly-Labeled Rand index scores segmentations appropriately in reference to qualitative performance and is more suitable than traditional quantitative metrics for scoring weakly-labeled data.