Taxonomy-Aware Continual Semantic Segmentation in Hyperbolic Spaces for Open-World Perception
This work addresses the challenge of updating semantic segmentation models with new classes while preventing forgetting in autonomous driving, though it is incremental as it builds on existing hyperbolic space methods.
The paper tackles the problem of class-incremental semantic segmentation in open-world scenarios by proposing TOPICS, which learns feature embeddings in hyperbolic space guided by taxonomy structures, achieving state-of-the-art performance on Cityscapes and Mapillary Vistas 2.0 benchmarks.
Semantic segmentation models are typically trained on a fixed set of classes, limiting their applicability in open-world scenarios. Class-incremental semantic segmentation aims to update models with emerging new classes while preventing catastrophic forgetting of previously learned ones. However, existing methods impose strict rigidity on old classes, reducing their effectiveness in learning new incremental classes. In this work, we propose Taxonomy-Oriented Poincaré-regularized Incremental-Class Segmentation (TOPICS) that learns feature embeddings in hyperbolic space following explicit taxonomy-tree structures. This supervision provides plasticity for old classes, updating ancestors based on new classes while integrating new classes at fitting positions. Additionally, we maintain implicit class relational constraints on the geometric basis of the Poincaré ball. This ensures that the latent space can continuously adapt to new constraints while maintaining a robust structure to combat catastrophic forgetting. We also establish eight realistic incremental learning protocols for autonomous driving scenarios, where novel classes can originate from known classes or the background. Extensive evaluations of TOPICS on the Cityscapes and Mapillary Vistas 2.0 benchmarks demonstrate that it achieves state-of-the-art performance. We make the code and trained models publicly available at http://topics.cs.uni-freiburg.de.