HiMODE: A Hybrid Monocular Omnidirectional Depth Estimation Model
This work addresses depth estimation for 360° surroundings, which is important for applications like robotics and VR, but appears incremental as it builds on existing hybrid architectures.
The paper tackles monocular omnidirectional depth estimation by proposing HiMODE, a hybrid CNN+Transformer model that addresses limitations in recovering small object details and data loss in ground-truth depth maps. It achieves state-of-the-art performance on three datasets (Stanford3D, Matterport3D, SunCG) with improved visual quality over ground-truth.
Monocular omnidirectional depth estimation is receiving considerable research attention due to its broad applications for sensing 360° surroundings. Existing approaches in this field suffer from limitations in recovering small object details and data lost during the ground-truth depth map acquisition. In this paper, a novel monocular omnidirectional depth estimation model, namely HiMODE is proposed based on a hybrid CNN+Transformer (encoder-decoder) architecture whose modules are efficiently designed to mitigate distortion and computational cost, without performance degradation. Firstly, we design a feature pyramid network based on the HNet block to extract high-resolution features near the edges. The performance is further improved, benefiting from a self and cross attention layer and spatial/temporal patches in the Transformer encoder and decoder, respectively. Besides, a spatial residual block is employed to reduce the number of parameters. By jointly passing the deep features extracted from an input image at each backbone block, along with the raw depth maps predicted by the transformer encoder-decoder, through a context adjustment layer, our model can produce resulting depth maps with better visual quality than the ground-truth. Comprehensive ablation studies demonstrate the significance of each individual module. Extensive experiments conducted on three datasets; Stanford3D, Matterport3D, and SunCG, demonstrate that HiMODE can achieve state-of-the-art performance for 360° monocular depth estimation.