Neural-Guided RANSAC: Learning Where to Sample Model Hypotheses
This work addresses the challenge of robust model fitting in computer vision, offering a principled learning approach that can be self-supervised and integrated with differentiable RANSAC, though it is incremental as it builds on the classic RANSAC algorithm.
The paper tackles the problem of improving model hypothesis search in robust optimization by introducing Neural-Guided RANSAC (NG-RANSAC), which uses learned prior information to increase the chance of finding outlier-free minimal sets, achieving superior or competitive results on tasks like epipolar geometry estimation, horizon line estimation, and camera re-localization compared to state-of-the-art methods.
We present Neural-Guided RANSAC (NG-RANSAC), an extension to the classic RANSAC algorithm from robust optimization. NG-RANSAC uses prior information to improve model hypothesis search, increasing the chance of finding outlier-free minimal sets. Previous works use heuristic side-information like hand-crafted descriptor distance to guide hypothesis search. In contrast, we learn hypothesis search in a principled fashion that lets us optimize an arbitrary task loss during training, leading to large improvements on classic computer vision tasks. We present two further extensions to NG-RANSAC. Firstly, using the inlier count itself as training signal allows us to train neural guidance in a self-supervised fashion. Secondly, we combine neural guidance with differentiable RANSAC to build neural networks which focus on certain parts of the input data and make the output predictions as good as possible. We evaluate NG-RANSAC on a wide array of computer vision tasks, namely estimation of epipolar geometry, horizon line estimation and camera re-localization. We achieve superior or competitive results compared to state-of-the-art robust estimators, including very recent, learned ones.