Learning Structure-from-Motion with Graph Attention Networks
This addresses the computer vision problem of SfM for faster and more efficient 3D reconstruction, though it is incremental as it builds on existing learning-based approaches.
The paper tackles the problem of learning Structure-from-Motion (SfM) by replacing traditional sub-problems with a graph attention network model that directly outputs camera poses and 3D keypoints from 2D inputs, achieving faster inference and outperforming learning-based methods while challenging COLMAP with lower runtime.
In this paper we tackle the problem of learning Structure-from-Motion (SfM) through the use of graph attention networks. SfM is a classic computer vision problem that is solved though iterative minimization of reprojection errors, referred to as Bundle Adjustment (BA), starting from a good initialization. In order to obtain a good enough initialization to BA, conventional methods rely on a sequence of sub-problems (such as pairwise pose estimation, pose averaging or triangulation) which provide an initial solution that can then be refined using BA. In this work we replace these sub-problems by learning a model that takes as input the 2D keypoints detected across multiple views, and outputs the corresponding camera poses and 3D keypoint coordinates. Our model takes advantage of graph neural networks to learn SfM-specific primitives, and we show that it can be used for fast inference of the reconstruction for new and unseen sequences. The experimental results show that the proposed model outperforms competing learning-based methods, and challenges COLMAP while having lower runtime. Our code is available at https://github.com/lucasbrynte/gasfm/.