SemHint-MD: Learning from Noisy Semantic Labels for Self-Supervised Monocular Depth Estimation
This work addresses a key bottleneck in self-supervised depth estimation for applications like autonomous driving and medical imaging, though it is incremental over prior methods.
The paper tackles the gradient-locality issue in self-supervised monocular depth estimation by using noisy semantic labels to guide the network out of local minima, achieving improved performance on the KITTI benchmark and endoscopic tissue tracking.
Without ground truth supervision, self-supervised depth estimation can be trapped in a local minimum due to the gradient-locality issue of the photometric loss. In this paper, we present a framework to enhance depth by leveraging semantic segmentation to guide the network to jump out of the local minimum. Prior works have proposed to share encoders between these two tasks or explicitly align them based on priors like the consistency between edges in the depth and segmentation maps. Yet, these methods usually require ground truth or high-quality pseudo labels, which may not be easily accessible in real-world applications. In contrast, we investigate self-supervised depth estimation along with a segmentation branch that is supervised with noisy labels provided by models pre-trained with limited data. We extend parameter sharing from the encoder to the decoder and study the influence of different numbers of shared decoder parameters on model performance. Also, we propose to use cross-task information to refine current depth and segmentation predictions to generate pseudo-depth and semantic labels for training. The advantages of the proposed method are demonstrated through extensive experiments on the KITTI benchmark and a downstream task for endoscopic tissue deformation tracking.