Towards 3D Object-Centric Feature Learning for Semantic Scene Completion
This addresses the need for more accurate semantic occupancy prediction in autonomous driving, but it is incremental as it builds on existing SSC methods by focusing on object-centric features.
The paper tackles the problem of fine-grained object-level details being overlooked in 3D Semantic Scene Completion, which causes semantic and geometric ambiguities in complex environments, and proposes Ocean, an object-centric framework that achieves state-of-the-art mIoU scores of 17.40 on SemanticKITTI and 20.28 on SSCBench-KITTI360.
Vision-based 3D Semantic Scene Completion (SSC) has received growing attention due to its potential in autonomous driving. While most existing approaches follow an ego-centric paradigm by aggregating and diffusing features over the entire scene, they often overlook fine-grained object-level details, leading to semantic and geometric ambiguities, especially in complex environments. To address this limitation, we propose Ocean, an object-centric prediction framework that decomposes the scene into individual object instances to enable more accurate semantic occupancy prediction. Specifically, we first employ a lightweight segmentation model, MobileSAM, to extract instance masks from the input image. Then, we introduce a 3D Semantic Group Attention module that leverages linear attention to aggregate object-centric features in 3D space. To handle segmentation errors and missing instances, we further design a Global Similarity-Guided Attention module that leverages segmentation features for global interaction. Finally, we propose an Instance-aware Local Diffusion module that improves instance features through a generative process and subsequently refines the scene representation in the BEV space. Extensive experiments on the SemanticKITTI and SSCBench-KITTI360 benchmarks demonstrate that Ocean achieves state-of-the-art performance, with mIoU scores of 17.40 and 20.28, respectively.