Continual Road-Scene Semantic Segmentation via Feature-Aligned Symmetric Multi-Modal Network
This addresses sensor reliability issues in safety-critical applications like autonomous driving, though it is incremental as it builds on existing multimodal segmentation methods.
The paper tackles the problem of multimodal semantic segmentation in real-world scenarios where sensor data may be missing, by proposing a feature-aligned symmetric network that works even with one modality absent, and introduces a continual learning scheme for autonomous driving, achieving impressive performances on the SemanticKITTI dataset.
State-of-the-art multimodal semantic segmentation strategies combining LiDAR and color data are usually designed on top of asymmetric information-sharing schemes and assume that both modalities are always available. This strong assumption may not hold in real-world scenarios, where sensors are prone to failure or can face adverse conditions that make the acquired information unreliable. This problem is exacerbated when continual learning scenarios are considered since they have stringent data reliability constraints. In this work, we re-frame the task of multimodal semantic segmentation by enforcing a tightly coupled feature representation and a symmetric information-sharing scheme, which allows our approach to work even when one of the input modalities is missing. We also introduce an ad-hoc class-incremental continual learning scheme, proving our approach's effectiveness and reliability even in safety-critical settings, such as autonomous driving. We evaluate our approach on the SemanticKITTI dataset, achieving impressive performances.