Generalizing to the Open World: Deep Visual Odometry with Online Adaptation
This addresses the domain adaptation problem for visual odometry in robotics and autonomous systems, offering an incremental improvement over existing self-supervised methods.
The paper tackles the problem of deep visual odometry networks failing in unseen environments due to domain gaps, by proposing an online adaptation framework that uses scene-agnostic geometry and Bayesian inference to continuously improve depth and pose estimation. The result is state-of-the-art generalization ability in self-supervised VO methods, as demonstrated in experiments like Cityscapes to KITTI and KITTI to TUM.
Despite learning-based visual odometry (VO) has shown impressive results in recent years, the pretrained networks may easily collapse in unseen environments. The large domain gap between training and testing data makes them difficult to generalize to new scenes. In this paper, we propose an online adaptation framework for deep VO with the assistance of scene-agnostic geometric computations and Bayesian inference. In contrast to learning-based pose estimation, our method solves pose from optical flow and depth while the single-view depth estimation is continuously improved with new observations by online learned uncertainties. Meanwhile, an online learned photometric uncertainty is used for further depth and pose optimization by a differentiable Gauss-Newton layer. Our method enables fast adaptation of deep VO networks to unseen environments in a self-supervised manner. Extensive experiments including Cityscapes to KITTI and outdoor KITTI to indoor TUM demonstrate that our method achieves state-of-the-art generalization ability among self-supervised VO methods.