GLPanoDepth: Global-to-Local Panoramic Depth Estimation
This work improves depth estimation for applications like VR and robotics by introducing a novel global-to-local strategy, though it is incremental as it builds on existing transformer and convolution methods.
The paper tackles panoramic depth estimation from monocular omnidirectional images by addressing the limitations of fully-convolutional networks in capturing global contexts and handling distortion, resulting in state-of-the-art performance.
In this paper, we propose a learning-based method for predicting dense depth values of a scene from a monocular omnidirectional image. An omnidirectional image has a full field-of-view, providing much more complete descriptions of the scene than perspective images. However, fully-convolutional networks that most current solutions rely on fail to capture rich global contexts from the panorama. To address this issue and also the distortion of equirectangular projection in the panorama, we propose Cubemap Vision Transformers (CViT), a new transformer-based architecture that can model long-range dependencies and extract distortion-free global features from the panorama. We show that cubemap vision transformers have a global receptive field at every stage and can provide globally coherent predictions for spherical signals. To preserve important local features, we further design a convolution-based branch in our pipeline (dubbed GLPanoDepth) and fuse global features from cubemap vision transformers at multiple scales. This global-to-local strategy allows us to fully exploit useful global and local features in the panorama, achieving state-of-the-art performance in panoramic depth estimation.