3D Geometry-Aware Semantic Labeling of Outdoor Street Scenes
This work addresses semantic labeling for outdoor street scenes, offering an incremental improvement by enhancing 3D geometry integration.
The paper tackled the problem of dense semantic image labeling by better exploiting 3D geometric information, proposing a method that uses 3D convolution in a residual connected 3D voxel top-down modulation network, which outperformed state-of-the-art methods on the Synthia and Cityscape datasets.
This paper is concerned with the problem of how to better exploit 3D geometric information for dense semantic image labeling. Existing methods often treat the available 3D geometry information (e.g., 3D depth-map) simply as an additional image channel besides the R-G-B color channels, and apply the same technique for RGB image labeling. In this paper, we demonstrate that directly performing 3D convolution in the framework of a residual connected 3D voxel top-down modulation network can lead to superior results. Specifically, we propose a 3D semantic labeling method to label outdoor street scenes whenever a dense depth map is available. Experiments on the "Synthia" and "Cityscape" datasets show our method outperforms the state-of-the-art methods, suggesting such a simple 3D representation is effective in incorporating 3D geometric information.