Not All Voxels Are Equal: Semantic Scene Completion from the Point-Voxel Perspective
This work improves SSC for 3D scene understanding by reducing computational costs, though it is incremental as it builds on existing voxel-based approaches.
The paper tackles the problem of Semantic Scene Completion (SSC) by addressing computational redundancy from visible empty voxels in existing methods, proposing a point-voxel aggregation network that achieves state-of-the-art results on two benchmarks with only depth images as input.
We revisit Semantic Scene Completion (SSC), a useful task to predict the semantic and occupancy representation of 3D scenes, in this paper. A number of methods for this task are always based on voxelized scene representations for keeping local scene structure. However, due to the existence of visible empty voxels, these methods always suffer from heavy computation redundancy when the network goes deeper, and thus limit the completion quality. To address this dilemma, we propose our novel point-voxel aggregation network for this task. Firstly, we transfer the voxelized scenes to point clouds by removing these visible empty voxels and adopt a deep point stream to capture semantic information from the scene efficiently. Meanwhile, a light-weight voxel stream containing only two 3D convolution layers preserves local structures of the voxelized scenes. Furthermore, we design an anisotropic voxel aggregation operator to fuse the structure details from the voxel stream into the point stream, and a semantic-aware propagation module to enhance the up-sampling process in the point stream by semantic labels. We demonstrate that our model surpasses state-of-the-arts on two benchmarks by a large margin, with only depth images as the input.