ThermalDiffusion: Visual-to-Thermal Image-to-Image Translation for Autonomous Navigation
This work addresses a data bottleneck for robotics and automation by augmenting datasets with synthetic thermal imagery, facilitating the adaptation of thermal cameras in autonomous systems.
The paper tackles the lack of thermal imagery data for autonomous navigation by proposing a method to generate synthetic thermal images from RGB images using conditional diffusion models, achieving results that enable tasks like scene segmentation and object detection in degraded environments.
Autonomous systems rely on sensors to estimate the environment around them. However, cameras, LiDARs, and RADARs have their own limitations. In nighttime or degraded environments such as fog, mist, or dust, thermal cameras can provide valuable information regarding the presence of objects of interest due to their heat signature. They make it easy to identify humans and vehicles that are usually at higher temperatures compared to their surroundings. In this paper, we focus on the adaptation of thermal cameras for robotics and automation, where the biggest hurdle is the lack of data. Several multi-modal datasets are available for driving robotics research in tasks such as scene segmentation, object detection, and depth estimation, which are the cornerstone of autonomous systems. However, they are found to be lacking in thermal imagery. Our paper proposes a solution to augment these datasets with synthetic thermal data to enable widespread and rapid adaptation of thermal cameras. We explore the use of conditional diffusion models to convert existing RGB images to thermal images using self-attention to learn the thermal properties of real-world objects.