Sequential Ensembling for Semantic Segmentation
This work addresses the need for more effective ensemble methods in semantic segmentation, which is crucial for applications like autonomous driving and mobile deployment, though it is incremental as it builds on existing ensembling and boosting ideas.
The paper tackles the problem of improving semantic segmentation by proposing a sequential ensembling method that outperforms naive ensemble baselines, achieving new state-of-the-art results on datasets like Cityscapes, ADE-20K, COCO-Stuff, and PASCAL-Context.
Ensemble approaches for deep-learning-based semantic segmentation remain insufficiently explored despite the proliferation of competitive benchmarks and downstream applications. In this work, we explore and benchmark the popular ensembling approach of combining predictions of multiple, independently-trained, state-of-the-art models at test time on popular datasets. Furthermore, we propose a novel method inspired by boosting to sequentially ensemble networks that significantly outperforms the naive ensemble baseline. Our approach trains a cascade of models conditioned on class probabilities predicted by the previous model as an additional input. A key benefit of this approach is that it allows for dynamic computation offloading, which helps deploy models on mobile devices. Our proposed novel ADaptive modulatiON (ADON) block allows spatial feature modulation at various layers using previous-stage probabilities. Our approach does not require sophisticated sample selection strategies during training and works with multiple neural architectures. We significantly improve over the naive ensemble baseline on challenging datasets such as Cityscapes, ADE-20K, COCO-Stuff, and PASCAL-Context and set a new state-of-the-art.