Kick Back & Relax: Learning to Reconstruct the World by Watching SlowTV
This addresses the problem of scaling depth estimation to complex, non-automotive environments for applications in robotics and AR/VR, though it is incremental in improving generalization through dataset expansion.
The authors tackled the problem of limited generalization in self-supervised monocular depth estimation by curating a large-scale SlowTV dataset from YouTube, containing 1.7M images from diverse environments, and trained a model that outperforms all existing SSL approaches and closes the gap on supervised state-of-the-art with zero-shot generalization to indoor/outdoor datasets.
Self-supervised monocular depth estimation (SS-MDE) has the potential to scale to vast quantities of data. Unfortunately, existing approaches limit themselves to the automotive domain, resulting in models incapable of generalizing to complex environments such as natural or indoor settings. To address this, we propose a large-scale SlowTV dataset curated from YouTube, containing an order of magnitude more data than existing automotive datasets. SlowTV contains 1.7M images from a rich diversity of environments, such as worldwide seasonal hiking, scenic driving and scuba diving. Using this dataset, we train an SS-MDE model that provides zero-shot generalization to a large collection of indoor/outdoor datasets. The resulting model outperforms all existing SSL approaches and closes the gap on supervised SoTA, despite using a more efficient architecture. We additionally introduce a collection of best-practices to further maximize performance and zero-shot generalization. This includes 1) aspect ratio augmentation, 2) camera intrinsic estimation, 3) support frame randomization and 4) flexible motion estimation. Code is available at https://github.com/jspenmar/slowtv_monodepth.