DRO: Deep Recurrent Optimizer for Video to Depth
This work addresses the video-to-depth estimation problem for computer vision applications, offering a novel optimization approach that outperforms previous methods, including supervised ones, in a self-supervised manner.
The paper tackles the video-to-depth problem by introducing a deep recurrent optimizer that alternately updates depth and camera poses to minimize a feature-metric cost, achieving state-of-the-art performance on KITTI and ScanNet datasets with improved efficiency in computation and memory.
There are increasing interests of studying the video-to-depth (V2D) problem with machine learning techniques. While earlier methods directly learn a mapping from images to depth maps and camera poses, more recent works enforce multi-view geometry constraints through optimization embedded in the learning framework. This paper presents a novel optimization method based on recurrent neural networks to further exploit the potential of neural networks in V2D. Specifically, our neural optimizer alternately updates the depth and camera poses through iterations to minimize a feature-metric cost, and two gated recurrent units iteratively improve the results by tracing historical information. Extensive experimental results demonstrate that our method outperforms previous methods and is more efficient in computation and memory consumption than cost-volume-based methods. In particular, our self-supervised method outperforms previous supervised methods on the KITTI and ScanNet datasets. Our source code is available at https://github.com/aliyun/dro-sfm.