Rescan: Inductive Instance Segmentation for Indoor RGBD Scans
This addresses the need for object tracking in applications like home robotics and AR/VR using sparse temporal data, representing an incremental improvement over existing methods.
The paper tackles the problem of inferring a temporal model of indoor scenes with semantic instance information from sparse RGBD rescans, and demonstrates that their algorithm outperforms state-of-the-art networks on a new benchmark.
In depth-sensing applications ranging from home robotics to AR/VR, it will be common to acquire 3D scans of interior spaces repeatedly at sparse time intervals (e.g., as part of regular daily use). We propose an algorithm that analyzes these "rescans" to infer a temporal model of a scene with semantic instance information. Our algorithm operates inductively by using the temporal model resulting from past observations to infer an instance segmentation of a new scan, which is then used to update the temporal model. The model contains object instance associations across time and thus can be used to track individual objects, even though there are only sparse observations. During experiments with a new benchmark for the new task, our algorithm outperforms alternate approaches based on state-of-the-art networks for semantic instance segmentation.