Robust Perceptual Night Vision in Thermal Colorization
This work addresses the challenge of night vision for applications like surveillance or autonomous driving by improving thermal-to-visible image colorization, though it appears incremental as it builds on existing methods with specific enhancements.
The paper tackles the ill-posed problem of transforming thermal infrared images into color visible images by proposing a deep learning method that maps thermal signatures to low-frequency visible representations and merges them with high-frequency thermal details, resulting in robust perceptual night vision images that preserve object appearance and image context compared to state-of-the-art methods.
Transforming a thermal infrared image into a robust perceptual colour Visible image is an ill-posed problem due to the differences in their spectral domains and in the objects' representations. Objects appear in one spectrum but not necessarily in the other, and the thermal signature of a single object may have different colours in its Visible representation. This makes a direct mapping from thermal to Visible images impossible and necessitates a solution that preserves texture captured in the thermal spectrum while predicting the possible colour for certain objects. In this work, a deep learning method to map the thermal signature from the thermal image's spectrum to a Visible representation in their low-frequency space is proposed. A pan-sharpening method is then used to merge the predicted low-frequency representation with the high-frequency representation extracted from the thermal image. The proposed model generates colour values consistent with the Visible ground truth when the object does not vary much in its appearance and generates averaged grey values in other cases. The proposed method shows robust perceptual night vision images in preserving the object's appearance and image context compared with the existing state-of-the-art.