The 8-Point Algorithm as an Inductive Bias for Relative Pose Prediction by ViTs
This work addresses the challenge of relative pose estimation for computer vision applications, offering a simpler and more effective solution compared to complex existing methods.
The paper tackles the problem of directly estimating relative pose between two images by introducing modifications to a Vision Transformer that align its computations with the Eight-Point Algorithm, resulting in competitive performance and substantial improvements over state-of-the-art methods, especially in limited data regimes.
We present a simple baseline for directly estimating the relative pose (rotation and translation, including scale) between two images. Deep methods have recently shown strong progress but often require complex or multi-stage architectures. We show that a handful of modifications can be applied to a Vision Transformer (ViT) to bring its computations close to the Eight-Point Algorithm. This inductive bias enables a simple method to be competitive in multiple settings, often substantially improving over the state of the art with strong performance gains in limited data regimes.