Real-time Indoor Scene Reconstruction with RGBD and Inertia Input
This work addresses the limitation of slow motion assumptions in camera motion estimation for SLAM, enabling more robust real-time indoor reconstruction for applications like robotics or AR.
The paper tackles the problem of 3D scene reconstruction under fast camera motions by proposing an end-to-end system that fuses RGBD and inertial data, using a novel geometry-aware patch deformation technique to improve feature matching accuracy, resulting in outperforming state-of-the-art solutions in such conditions.
Camera motion estimation is a key technique for 3D scene reconstruction and Simultaneous localization and mapping (SLAM). To make it be feasibly achieved, previous works usually assume slow camera motions, which limits its usage in many real cases. We propose an end-to-end 3D reconstruction system which combines color, depth and inertial measurements to achieve robust reconstruction with fast sensor motions. Our framework extends Kalman filter to fuse the three kinds of information and involve an iterative method to jointly optimize feature correspondences, camera poses and scene geometry. We also propose a novel geometry-aware patch deformation technique to adapt the feature appearance in image domain, leading to a more accurate feature matching under fast camera motions. Experiments show that our patch deformation method improves the accuracy of feature tracking, and our 3D reconstruction outperforms the state-of-the-art solutions under fast camera motions.