Seamless Scene Segmentation
This work addresses the challenge of seamless scene segmentation for autonomous driving and urban scene analysis, offering an incremental improvement over existing methods by combining segmentation and detection into a panoptic format.
The authors tackled the problem of achieving consistent semantic segmentation and detection in street-level scenes by proposing a novel CNN-based architecture that integrates multi-scale features with contextual information, resulting in state-of-the-art performance on Cityscapes, Indian Driving Dataset, and Mapillary Vistas datasets.
In this work we introduce a novel, CNN-based architecture that can be trained end-to-end to deliver seamless scene segmentation results. Our goal is to predict consistent semantic segmentation and detection results by means of a panoptic output format, going beyond the simple combination of independently trained segmentation and detection models. The proposed architecture takes advantage of a novel segmentation head that seamlessly integrates multi-scale features generated by a Feature Pyramid Network with contextual information conveyed by a light-weight DeepLab-like module. As additional contribution we review the panoptic metric and propose an alternative that overcomes its limitations when evaluating non-instance categories. Our proposed network architecture yields state-of-the-art results on three challenging street-level datasets, i.e. Cityscapes, Indian Driving Dataset and Mapillary Vistas.