Unpaired Thermal to Visible Spectrum Transfer using Adversarial Training
This addresses the usability issue of TIR images for human perception in computer vision applications, though it is incremental as it adapts existing GAN techniques to a specific data alignment problem.
The paper tackles the problem of transforming thermal infrared (TIR) images into realistic visible spectrum (VIS) images, which is challenging due to imperfect alignment in existing datasets, by using an unsupervised GAN on unpaired data; it achieves significantly more realistic and sharp VIS images than state-of-the-art supervised methods on the KAIST-MS dataset and shows strong generalization to new environments.
Thermal Infrared (TIR) cameras are gaining popularity in many computer vision applications due to their ability to operate under low-light conditions. Images produced by TIR cameras are usually difficult for humans to perceive visually, which limits their usability. Several methods in the literature were proposed to address this problem by transforming TIR images into realistic visible spectrum (VIS) images. However, existing TIR-VIS datasets suffer from imperfect alignment between TIR-VIS image pairs which degrades the performance of supervised methods. We tackle this problem by learning this transformation using an unsupervised Generative Adversarial Network (GAN) which trains on unpaired TIR and VIS images. When trained and evaluated on KAIST-MS dataset, our proposed methods was shown to produce significantly more realistic and sharp VIS images than the existing state-of-the-art supervised methods. In addition, our proposed method was shown to generalize very well when evaluated on a new dataset of new environments.