Multimodal Signal Processing For Thermo-Visible-Lidar Fusion In Real-time 3D Semantic Mapping
This work addresses the need for real-time 3D semantic mapping with thermal data in complex environments, specifically for autonomous robots in disaster assessment and industrial maintenance, representing an incremental improvement by integrating existing modalities.
The paper tackles the problem of enhancing 3D point cloud maps with thermal information for autonomous robot navigation by fusing visible and infrared images, projecting LiDAR point clouds onto this stream, and segmenting heat sources to identify high-temperature targets, resulting in maps with accurate geometry and critical semantic understanding for applications like disaster assessment and industrial maintenance.
In complex environments, autonomous robot navigation and environmental perception pose higher requirements for SLAM technology. This paper presents a novel method for semantically enhancing 3D point cloud maps with thermal information. By first performing pixel-level fusion of visible and infrared images, the system projects real-time LiDAR point clouds onto this fused image stream. It then segments heat source features in the thermal channel to instantly identify high temperature targets and applies this temperature information as a semantic layer on the final 3D map. This approach generates maps that not only have accurate geometry but also possess a critical semantic understanding of the environment, making it highly valuable for specific applications like rapid disaster assessment and industrial preventive maintenance.