M3DMap: Object-aware Multimodal 3D Mapping for Dynamic Environments
This work addresses the problem of dynamic 3D mapping for robotics and autonomous transportation, but it appears incremental as it builds on existing approaches with a new modular framework.
The paper tackles the challenge of creating multimodal 3D maps in dynamic environments by proposing a taxonomy of methods and introducing M3DMap, a modular method for object-aware construction, which integrates neural segmentation, odometry, and data retrieval to improve tasks like 3D object grounding and mobile manipulation.
3D mapping in dynamic environments poses a challenge for modern researchers in robotics and autonomous transportation. There are no universal representations for dynamic 3D scenes that incorporate multimodal data such as images, point clouds, and text. This article takes a step toward solving this problem. It proposes a taxonomy of methods for constructing multimodal 3D maps, classifying contemporary approaches based on scene types and representations, learning methods, and practical applications. Using this taxonomy, a brief structured analysis of recent methods is provided. The article also describes an original modular method called M3DMap, designed for object-aware construction of multimodal 3D maps for both static and dynamic scenes. It consists of several interconnected components: a neural multimodal object segmentation and tracking module; an odometry estimation module, including trainable algorithms; a module for 3D map construction and updating with various implementations depending on the desired scene representation; and a multimodal data retrieval module. The article highlights original implementations of these modules and their advantages in solving various practical tasks, from 3D object grounding to mobile manipulation. Additionally, it presents theoretical propositions demonstrating the positive effect of using multimodal data and modern foundational models in 3D mapping methods. Details of the taxonomy and method implementation are available at https://yuddim.github.io/M3DMap.