DNS SLAM: Dense Neural Semantic-Informed SLAM
This work addresses the challenge of detailed and semantic-aware mapping in SLAM for robotics and AR/VR applications, representing an incremental improvement with novel integration of semantics.
The authors tackled the problem of oversmoothed reconstructions in neural SLAM for complex real-world scenes by introducing DNS SLAM, a hybrid neural RGB-D semantic SLAM method that achieves state-of-the-art performance on synthetic and real-world data with commendable operational speed.
In recent years, coordinate-based neural implicit representations have shown promising results for the task of Simultaneous Localization and Mapping (SLAM). While achieving impressive performance on small synthetic scenes, these methods often suffer from oversmoothed reconstructions, especially for complex real-world scenes. In this work, we introduce DNS SLAM, a novel neural RGB-D semantic SLAM approach featuring a hybrid representation. Relying only on 2D semantic priors, we propose the first semantic neural SLAM method that trains class-wise scene representations while providing stable camera tracking at the same time. Our method integrates multi-view geometry constraints with image-based feature extraction to improve appearance details and to output color, density, and semantic class information, enabling many downstream applications. To further enable real-time tracking, we introduce a lightweight coarse scene representation which is trained in a self-supervised manner in latent space. Our experimental results achieve state-of-the-art performance on both synthetic data and real-world data tracking while maintaining a commendable operational speed on off-the-shelf hardware. Further, our method outputs class-wise decomposed reconstructions with better texture capturing appearance and geometric details.