Feature-metric Loss for Self-supervised Learning of Depth and Egomotion
This work addresses optimization challenges in self-supervised learning for depth and egomotion estimation, which is important for robotics and autonomous systems, and is incremental as it builds on existing photometric loss methods.
The paper tackles the problem of self-supervised depth and egomotion estimation by addressing issues with photometric loss landscapes, such as plateaus in textureless regions and local minima, and proposes a feature-metric loss that improves state-of-the-art depth estimation on KITTI from 0.885 to 0.925 in δ1 metric and significantly outperforms previous methods for visual odometry.
Photometric loss is widely used for self-supervised depth and egomotion estimation. However, the loss landscapes induced by photometric differences are often problematic for optimization, caused by plateau landscapes for pixels in textureless regions or multiple local minima for less discriminative pixels. In this work, feature-metric loss is proposed and defined on feature representation, where the feature representation is also learned in a self-supervised manner and regularized by both first-order and second-order derivatives to constrain the loss landscapes to form proper convergence basins. Comprehensive experiments and detailed analysis via visualization demonstrate the effectiveness of the proposed feature-metric loss. In particular, our method improves state-of-the-art methods on KITTI from 0.885 to 0.925 measured by $δ_1$ for depth estimation, and significantly outperforms previous method for visual odometry.