Self-Supervised Depth Estimation with Isometric-Self-Sample-Based Learning
This addresses a bottleneck in self-supervised depth estimation for computer vision applications, though it appears incremental as it builds on existing models.
The paper tackles the problem of handling dynamic regions in self-supervised depth estimation by proposing an isometric self-sample-based learning method that generates synthetic images complying with static scene assumptions, resulting in consistent performance improvements across multiple datasets including KITTI, Make3D, and NYUv2.
Managing the dynamic regions in the photometric loss formulation has been a main issue for handling the self-supervised depth estimation problem. Most previous methods have alleviated this issue by removing the dynamic regions in the photometric loss formulation based on the masks estimated from another module, making it difficult to fully utilize the training images. In this paper, to handle this problem, we propose an isometric self-sample-based learning (ISSL) method to fully utilize the training images in a simple yet effective way. The proposed method provides additional supervision during training using self-generated images that comply with pure static scene assumption. Specifically, the isometric self-sample generator synthesizes self-samples for each training image by applying random rigid transformations on the estimated depth. Thus both the generated self-samples and the corresponding training image always follow the static scene assumption. We show that plugging our ISSL module into several existing models consistently improves the performance by a large margin. In addition, it also boosts the depth accuracy over different types of scene, i.e., outdoor scenes (KITTI and Make3D) and indoor scene (NYUv2), validating its high effectiveness.