SGFormer: Spherical Geometry Transformer for 360 Depth Estimation
This addresses depth estimation challenges in 360-degree imaging for applications like VR and robotics, representing an incremental improvement with a novel method for a known bottleneck.
The paper tackled the problem of panoramic distortion in 360 depth estimation by proposing SGFormer, a spherical geometry transformer that integrates spherical geometric priors, resulting in superior performance over state-of-the-art methods on popular benchmarks.
Panoramic distortion poses a significant challenge in 360 depth estimation, particularly pronounced at the north and south poles. Existing methods either adopt a bi-projection fusion strategy to remove distortions or model long-range dependencies to capture global structures, which can result in either unclear structure or insufficient local perception. In this paper, we propose a spherical geometry transformer, named SGFormer, to address the above issues, with an innovative step to integrate spherical geometric priors into vision transformers. To this end, we retarget the transformer decoder to a spherical prior decoder (termed SPDecoder), which endeavors to uphold the integrity of spherical structures during decoding. Concretely, we leverage bipolar re-projection, circular rotation, and curve local embedding to preserve the spherical characteristics of equidistortion, continuity, and surface distance, respectively. Furthermore, we present a query-based global conditional position embedding to compensate for spatial structure at varying resolutions. It not only boosts the global perception of spatial position but also sharpens the depth structure across different patches. Finally, we conduct extensive experiments on popular benchmarks, demonstrating our superiority over state-of-the-art solutions.