Towards Accurate Single Panoramic 3D Detection: A Semantic Gaussian Centric Approach
This work addresses the challenge of accurate 3D detection from single panoramic images, which is crucial for autonomous driving and robotics, but the improvement is incremental over existing approaches.
PanoGSDet achieves state-of-the-art 3D object detection in panoramic imagery by using continuous semantic 3D Gaussians instead of discrete grids, outperforming prior methods on the Structured3D dataset.
Three-dimensional object detection in panoramic imagery is crucial for comprehensive scene understanding, yet accurately mapping 2D features to 3D remains a significant challenge. Prevailing methods often project 2D features onto discrete 3D grids, which break geometric continuity and limit representation efficiency. To overcome this limitation, this paper proposes PanoGSDet, a monocular panoramic 3D detection framework built upon continuous semantic 3D Gaussian representations. The proposed framework comprises a panoramic depth estimation component and a semantic Gaussian component. The panoramic depth estimation component extracts the equirectangular semantic and depth features from the monocular panorama input. The semantic Gaussian component includes a semantic Gaussian lifting module that projects spherical features into 3D semantic Gaussians, a semantic Gaussian optimization module that refines these semantic Gaussians, and a Gaussian guided prediction head that generates 3D bounding boxes from optimized Gaussian representations. Extensive experiments on the Structured3D dataset demonstrate that our method significantly outperforms existing methods.