SAMSelect: A Spectral Index Search for Marine Debris Visualization using Segment Anything
This work addresses the challenge for marine scientists who visually interpret marine debris imagery, though it appears incremental as it builds on existing segmentation methods for a specific domain application.
The researchers tackled the problem of visualizing floating marine debris in Sentinel-2 imagery, which is difficult due to compositional heterogeneity, by proposing SAMSelect, an algorithm that selects band or index combinations achieving the best classification accuracy through the Segment Anything Model, revealing new combinations like a normalized difference index of B8 and B2 that demonstrated improved performance compared to literature-based indices.
This work proposes SAMSelect, an algorithm to obtain a salient three-channel visualization for multispectral images. We develop SAMSelect and show its use for marine scientists visually interpreting floating marine debris in Sentinel-2 imagery. These debris are notoriously difficult to visualize due to their compositional heterogeneity in medium-resolution imagery. Out of these difficulties, a visual interpretation of imagery showing marine debris remains a common practice by domain experts, who select bands and spectral indices on a case-by-case basis informed by common practices and heuristics. SAMSelect selects the band or index combination that achieves the best classification accuracy on a small annotated dataset through the Segment Anything Model. Its central assumption is that the three-channel visualization achieves the most accurate segmentation results also provide good visual information for photo-interpretation. We evaluate SAMSelect in three Sentinel-2 scenes containing generic marine debris in Accra, Ghana, and Durban, South Africa, and deployed plastic targets from the Plastic Litter Project. This reveals the potential of new previously unused band combinations (e.g., a normalized difference index of B8, B2), which demonstrate improved performance compared to literature-based indices. We describe the algorithm in this paper and provide an open-source code repository that will be helpful for domain scientists doing visual photo interpretation, especially in the marine field.