InstanceBEV: Unifying Instance and BEV Representation for 3D Panoptic Segmentation
This work addresses efficiency and integration issues in 3D panoptic segmentation for autonomous driving systems, representing an incremental improvement over existing BEV methods.
The paper tackles the challenge of large feature spaces in BEV-based 3D perception for autonomous driving by proposing InstanceBEV, which unifies instance and BEV representations to enable efficient global attention modeling and multi-task learning. It achieves a RayPQ of 15.3 and RayIoU of 38.2 on the OCC3D-nuScenes dataset, surpassing SparseOcc by 9.3% and 10.7% respectively.
BEV-based 3D perception has emerged as a focal point of research in end-to-end autonomous driving. However, existing BEV approaches encounter significant challenges due to the large feature space, complicating efficient modeling and hindering effective integration of global attention mechanisms. We propose a novel modeling strategy, called InstanceBEV, that synergistically combines the strengths of both map-centric approaches and object-centric approaches. Our method effectively extracts instance-level features within the BEV features, facilitating the implementation of global attention modeling in a highly compressed feature space, thereby addressing the efficiency challenges inherent in map-centric global modeling. Furthermore, our approach enables effective multi-task learning without introducing additional module. We validate the efficiency and accuracy of the proposed model through predicting occupancy, achieving 3D occupancy panoptic segmentation by combining instance information. Experimental results on the OCC3D-nuScenes dataset demonstrate that InstanceBEV, utilizing only 8 frames, achieves a RayPQ of 15.3 and a RayIoU of 38.2. This surpasses SparseOcc's RayPQ by 9.3% and RayIoU by 10.7%, showcasing the effectiveness of multi-task synergy.