Region-Aware Metric Learning for Open World Semantic Segmentation via Meta-Channel Aggregation
This addresses the problem of segmenting anomalies and learning new objects in dynamic environments for applications like autonomous driving, but it is incremental as it builds on existing metric learning methods.
The paper tackles open-world semantic segmentation by proposing region-aware metric learning (RAML) with a meta-channel aggregation module to improve anomaly region segmentation and incremental learning for out-of-distribution objects, achieving state-of-the-art performance on datasets like Lost And Found and CityScapes.
As one of the most challenging and practical segmentation tasks, open-world semantic segmentation requires the model to segment the anomaly regions in the images and incrementally learn to segment out-of-distribution (OOD) objects, especially under a few-shot condition. The current state-of-the-art (SOTA) method, Deep Metric Learning Network (DMLNet), relies on pixel-level metric learning, with which the identification of similar regions having different semantics is difficult. Therefore, we propose a method called region-aware metric learning (RAML), which first separates the regions of the images and generates region-aware features for further metric learning. RAML improves the integrity of the segmented anomaly regions. Moreover, we propose a novel meta-channel aggregation (MCA) module to further separate anomaly regions, forming high-quality sub-region candidates and thereby improving the model performance for OOD objects. To evaluate the proposed RAML, we have conducted extensive experiments and ablation studies on Lost And Found and Road Anomaly datasets for anomaly segmentation and the CityScapes dataset for incremental few-shot learning. The results show that the proposed RAML achieves SOTA performance in both stages of open world segmentation. Our code and appendix are available at https://github.com/czifan/RAML.