CapHDR2IR: Caption-Driven Transfer from Visible Light to Infrared Domain
This work improves infrared image synthesis for applications in fields like surveillance or medical imaging where hardware limitations restrict IR sensor use, though it is incremental as it builds on existing domain transfer methods.
The paper tackles the problem of synthesizing infrared images from visible light by addressing fidelity loss and inconsistencies due to limited dynamic range and lack of contextual awareness, proposing CapHDR2IR which uses HDR images and vision-language models to achieve state-of-the-art performance on the HDRT dataset.
Infrared (IR) imaging offers advantages in several fields due to its unique ability of capturing content in extreme light conditions. However, the demanding hardware requirements of high-resolution IR sensors limit its widespread application. As an alternative, visible light can be used to synthesize IR images but this causes a loss of fidelity in image details and introduces inconsistencies due to lack of contextual awareness of the scene. This stems from a combination of using visible light with a standard dynamic range, especially under extreme lighting, and a lack of contextual awareness can result in pseudo-thermal-crossover artifacts. This occurs when multiple objects with similar temperatures appear indistinguishable in the training data, further exacerbating the loss of fidelity. To solve this challenge, this paper proposes CapHDR2IR, a novel framework incorporating vision-language models using high dynamic range (HDR) images as inputs to generate IR images. HDR images capture a wider range of luminance variations, ensuring reliable IR image generation in different light conditions. Additionally, a dense caption branch integrates semantic understanding, resulting in more meaningful and discernible IR outputs. Extensive experiments on the HDRT dataset show that the proposed CapHDR2IR achieves state-of-the-art performance compared with existing general domain transfer methods and those tailored for visible-to-infrared image translation.