Web Stereo Video Supervision for Depth Prediction from Dynamic Scenes
This addresses depth estimation in dynamic scenes with non-rigid objects, which is important for applications like robotics and AR, but is incremental as it builds on existing data-driven methods with a new dataset.
The paper tackles depth prediction from monocular videos containing non-rigid objects like people by introducing a new stereo video dataset scraped from the web, using it to train a method with a loss function that handles unknown camera parameters. The approach shows improved generalization to natural scenes on benchmarks like SINTEL and KITTI.
We present a fully data-driven method to compute depth from diverse monocular video sequences that contain large amounts of non-rigid objects, e.g., people. In order to learn reconstruction cues for non-rigid scenes, we introduce a new dataset consisting of stereo videos scraped in-the-wild. This dataset has a wide variety of scene types, and features large amounts of nonrigid objects, especially people. From this, we compute disparity maps to be used as supervision to train our approach. We propose a loss function that allows us to generate a depth prediction even with unknown camera intrinsics and stereo baselines in the dataset. We validate the use of large amounts of Internet video by evaluating our method on existing video datasets with depth supervision, including SINTEL, and KITTI, and show that our approach generalizes better to natural scenes.