Layered Interpretation of Street View Images
This work addresses scene understanding for autonomous driving systems, representing an incremental improvement over existing methods like stix-mantics.
The authors tackled the problem of encoding depth and semantic information from street view images for autonomous driving by proposing a 4-layer model, which outperformed competing approaches on the Daimler urban scene segmentation dataset with a processing speed of about 9 fps on a GPU.
We propose a layered street view model to encode both depth and semantic information on street view images for autonomous driving. Recently, stixels, stix-mantics, and tiered scene labeling methods have been proposed to model street view images. We propose a 4-layer street view model, a compact representation over the recently proposed stix-mantics model. Our layers encode semantic classes like ground, pedestrians, vehicles, buildings, and sky in addition to the depths. The only input to our algorithm is a pair of stereo images. We use a deep neural network to extract the appearance features for semantic classes. We use a simple and an efficient inference algorithm to jointly estimate both semantic classes and layered depth values. Our method outperforms other competing approaches in Daimler urban scene segmentation dataset. Our algorithm is massively parallelizable, allowing a GPU implementation with a processing speed about 9 fps.