Volume-based Semantic Labeling with Signed Distance Functions
This work addresses the challenge of creating semantically rich 3D maps for robotics or AR/VR applications, representing a novel integration but incremental in combining existing techniques.
The paper tackles the problem of integrating semantic segmentation with dense SLAM to produce semantically labeled 3D reconstructions from RGB-D images, achieving the first such method validated on a public dataset with ground truth and noisy labels.
Research works on the two topics of Semantic Segmentation and SLAM (Simultaneous Localization and Mapping) have been following separate tracks. Here, we link them quite tightly by delineating a category label fusion technique that allows for embedding semantic information into the dense map created by a volume-based SLAM algorithm such as KinectFusion. Accordingly, our approach is the first to provide a semantically labeled dense reconstruction of the environment from a stream of RGB-D images. We validate our proposal using a publicly available semantically annotated RGB-D dataset and a) employing ground truth labels, b) corrupting such annotations with synthetic noise, c) deploying a state of the art semantic segmentation algorithm based on Convolutional Neural Networks.