Learning Less is More - 6D Camera Localization via 3D Surface Regression
This work addresses camera localization for applications like autonomous driving and augmented reality, presenting an incremental improvement by simplifying the pipeline.
The paper tackles the problem of predicting 6D camera pose from a single RGB image by showing that learning only a single component—a fully convolutional neural network for dense scene coordinate regression—is sufficient, achieving state-of-the-art accuracy on indoor and outdoor datasets without requiring a 3D model during training.
Popular research areas like autonomous driving and augmented reality have renewed the interest in image-based camera localization. In this work, we address the task of predicting the 6D camera pose from a single RGB image in a given 3D environment. With the advent of neural networks, previous works have either learned the entire camera localization process, or multiple components of a camera localization pipeline. Our key contribution is to demonstrate and explain that learning a single component of this pipeline is sufficient. This component is a fully convolutional neural network for densely regressing so-called scene coordinates, defining the correspondence between the input image and the 3D scene space. The neural network is prepended to a new end-to-end trainable pipeline. Our system is efficient, highly accurate, robust in training, and exhibits outstanding generalization capabilities. It exceeds state-of-the-art consistently on indoor and outdoor datasets. Interestingly, our approach surpasses existing techniques even without utilizing a 3D model of the scene during training, since the network is able to discover 3D scene geometry automatically, solely from single-view constraints.