Semi-Supervised Semantic Depth Estimation using Symbiotic Transformer and NearFarMix Augmentation
This work addresses the problem of limited semantic information in depth estimation for applications like robotics and autonomous vehicles, representing an incremental improvement over existing methods.
The paper tackles the challenge of integrating semantics with depth estimation in computer vision by proposing a semi-supervised strategy with a Depth Semantics Symbiosis module and NearFarMix augmentation, achieving superior performance on NYU-Depth-V2 and KITTI datasets.
In computer vision, depth estimation is crucial for domains like robotics, autonomous vehicles, augmented reality, and virtual reality. Integrating semantics with depth enhances scene understanding through reciprocal information sharing. However, the scarcity of semantic information in datasets poses challenges. Existing convolutional approaches with limited local receptive fields hinder the full utilization of the symbiotic potential between depth and semantics. This paper introduces a dataset-invariant semi-supervised strategy to address the scarcity of semantic information. It proposes the Depth Semantics Symbiosis module, leveraging the Symbiotic Transformer for achieving comprehensive mutual awareness by information exchange within both local and global contexts. Additionally, a novel augmentation, NearFarMix is introduced to combat overfitting and compensate both depth-semantic tasks by strategically merging regions from two images, generating diverse and structurally consistent samples with enhanced control. Extensive experiments on NYU-Depth-V2 and KITTI datasets demonstrate the superiority of our proposed techniques in indoor and outdoor environments.