Voxel-Aggregated Feature Synthesis: Efficient Dense Mapping for Simulated 3D Reasoning
This addresses the problem of impractical computational requirements for embodied agent research in simulation, though it appears incremental as it builds on existing dense 3D mapping with a simulator-based optimization.
The paper tackles the computational inefficiency of dense 3D mapping algorithms by introducing Voxel-Aggregated Feature Synthesis (VAFS), which reduces feature embedding from RGBD frames to scene objects, achieving an order of magnitude faster computation while exceeding prior accuracy and speed.
We address the issue of the exploding computational requirements of recent State-of-the-art (SOTA) open set multimodel 3D mapping (dense 3D mapping) algorithms and present Voxel-Aggregated Feature Synthesis (VAFS), a novel approach to dense 3D mapping in simulation. Dense 3D mapping involves segmenting and embedding sequential RGBD frames which are then fused into 3D. This leads to redundant computation as the differences between frames are small but all are individually segmented and embedded. This makes dense 3D mapping impractical for research involving embodied agents in which the environment, and thus the mapping, must be modified with regularity. VAFS drastically reduces this computation by using the segmented point cloud computed by a simulator's physics engine and synthesizing views of each region. This reduces the number of features to embed from the number of captured RGBD frames to the number of objects in the scene, effectively allowing a "ground truth" semantic map to be computed an order of magnitude faster than traditional methods. We test the resulting representation by assessing the IoU scores of semantic queries for different objects in the simulated scene, and find that VAFS exceeds the accuracy and speed of prior dense 3D mapping techniques.