Open-World Semantic Segmentation Including Class Similarity
This addresses the problem of handling novel objects in real-world environments for autonomous systems like vehicles, representing a novel method rather than incremental improvement.
The paper tackles open-world semantic segmentation where objects not seen during training must be identified, proposing an approach that performs accurate closed-world segmentation while identifying new categories without additional training data and providing similarity measures to known classes. The model achieves state-of-the-art results on known classes and anomaly segmentation, and can distinguish between different unknown classes.
Interpreting camera data is key for autonomously acting systems, such as autonomous vehicles. Vision systems that operate in real-world environments must be able to understand their surroundings and need the ability to deal with novel situations. This paper tackles open-world semantic segmentation, i.e., the variant of interpreting image data in which objects occur that have not been seen during training. We propose a novel approach that performs accurate closed-world semantic segmentation and, at the same time, can identify new categories without requiring any additional training data. Our approach additionally provides a similarity measure for every newly discovered class in an image to a known category, which can be useful information in downstream tasks such as planning or mapping. Through extensive experiments, we show that our model achieves state-of-the-art results on classes known from training data as well as for anomaly segmentation and can distinguish between different unknown classes.