BA-Net: Dense Bundle Adjustment Network
This addresses the challenging dense 3D reconstruction problem for computer vision applications, representing an incremental improvement by combining domain knowledge with deep learning.
The paper tackles the dense structure-from-motion problem by introducing a differentiable network that enforces multi-view geometry constraints via feature-metric bundle adjustment and learns features and basis depth maps, achieving success in experiments on large-scale real data.
This paper introduces a network architecture to solve the structure-from-motion (SfM) problem via feature-metric bundle adjustment (BA), which explicitly enforces multi-view geometry constraints in the form of feature-metric error. The whole pipeline is differentiable so that the network can learn suitable features that make the BA problem more tractable. Furthermore, this work introduces a novel depth parameterization to recover dense per-pixel depth. The network first generates several basis depth maps according to the input image and optimizes the final depth as a linear combination of these basis depth maps via feature-metric BA. The basis depth maps generator is also learned via end-to-end training. The whole system nicely combines domain knowledge (i.e. hard-coded multi-view geometry constraints) and deep learning (i.e. feature learning and basis depth maps learning) to address the challenging dense SfM problem. Experiments on large scale real data prove the success of the proposed method.