Automated Label Unification for Multi-Dataset Semantic Segmentation with GNNs
This addresses the efficiency and effectiveness of multi-dataset training for semantic segmentation, reducing the need for manual reannotation, but it is incremental as it builds on existing multi-dataset approaches.
The paper tackles the problem of label space conflicts when training semantic segmentation models on multiple datasets by proposing an automated label unification method using graph neural networks, resulting in state-of-the-art performance on the WildDash 2 benchmark and outperforming other methods across seven datasets.
Deep supervised models possess significant capability to assimilate extensive training data, thereby presenting an opportunity to enhance model performance through training on multiple datasets. However, conflicts arising from different label spaces among datasets may adversely affect model performance. In this paper, we propose a novel approach to automatically construct a unified label space across multiple datasets using graph neural networks. This enables semantic segmentation models to be trained simultaneously on multiple datasets, resulting in performance improvements. Unlike existing methods, our approach facilitates seamless training without the need for additional manual reannotation or taxonomy reconciliation. This significantly enhances the efficiency and effectiveness of multi-dataset segmentation model training. The results demonstrate that our method significantly outperforms other multi-dataset training methods when trained on seven datasets simultaneously, and achieves state-of-the-art performance on the WildDash 2 benchmark.