UniFuse: Unidirectional Fusion for 360$^{\circ}$ Panorama Depth Estimation
This work provides a more efficient and effective method for 360-degree panorama depth estimation, which is important for applications requiring a full field-of-view understanding of environments, such as robotics or virtual reality.
This paper addresses the challenge of depth estimation from 360-degree panoramas, which suffer from distortion in equirectangular projection or discontinuities in cubemap projection. The authors propose UniFuse, a unidirectional fusion framework that feeds cubemap features into equirectangular features only during the decoding stage, achieving state-of-the-art performance on four popular datasets.
Learning depth from spherical panoramas is becoming a popular research topic because a panorama has a full field-of-view of the environment and provides a relatively complete description of a scene. However, applying well-studied CNNs for perspective images to the standard representation of spherical panoramas, i.e., the equirectangular projection, is suboptimal, as it becomes distorted towards the poles. Another representation is the cubemap projection, which is distortion-free but discontinued on edges and limited in the field-of-view. This paper introduces a new framework to fuse features from the two projections, unidirectionally feeding the cubemap features to the equirectangular features only at the decoding stage. Unlike the recent bidirectional fusion approach operating at both the encoding and decoding stages, our fusion scheme is much more efficient. Besides, we also designed a more effective fusion module for our fusion scheme. Experiments verify the effectiveness of our proposed fusion strategy and module, and our model achieves state-of-the-art performance on four popular datasets. Additional experiments show that our model also has the advantages of model complexity and generalization capability.The code is available at https://github.com/alibaba/UniFuse-Unidirectional-Fusion.