NEWTON: Neural View-Centric Mapping for On-the-Fly Large-Scale SLAM
This work solves the problem of enabling robust, real-time neural SLAM for large-scale environments with dynamic camera pose corrections, which is incremental as it builds on existing neural field-based SLAM by introducing a view-centric approach.
The paper tackles the problem of real-time large-scale SLAM by addressing the limitations of world-centric neural field representations, which require static prior information and struggle with camera pose drift and loop closures; the proposed view-centric method, NEWTON, dynamically constructs neural fields based on run-time observations and outperforms existing systems in large-scale scenes with pose updates.
Neural field-based 3D representations have recently been adopted in many areas including SLAM systems. Current neural SLAM or online mapping systems lead to impressive results in the presence of simple captures, but they rely on a world-centric map representation as only a single neural field model is used. To define such a world-centric representation, accurate and static prior information about the scene, such as its boundaries and initial camera poses, are required. However, in real-time and on-the-fly scene capture applications, this prior knowledge cannot be assumed as fixed or static, since it dynamically changes and it is subject to significant updates based on run-time observations. Particularly in the context of large-scale mapping, significant camera pose drift is inevitable, necessitating the correction via loop closure. To overcome this limitation, we propose NEWTON, a view-centric mapping method that dynamically constructs neural fields based on run-time observation. In contrast to prior works, our method enables camera pose updates using loop closures and scene boundary updates by representing the scene with multiple neural fields, where each is defined in a local coordinate system of a selected keyframe. The experimental results demonstrate the superior performance of our method over existing world-centric neural field-based SLAM systems, in particular for large-scale scenes subject to camera pose updates.