Visible and infrared self-supervised fusion trained on a single example
This addresses the need for efficient multispectral image fusion in applications like dehazing or object detection, offering a lightweight alternative to data-intensive methods.
The paper tackles the problem of fusing visible (RGB) and near-infrared (NIR) images by training a convolutional neural network using self-supervised learning on just a single example, achieving results comparable to or better than state-of-the-art methods without heavy training or large datasets.
Multispectral imaging is an important task of image processing and computer vision, which is especially relevant to applications such as dehazing or object detection. With the development of the RGBT (RGB & Thermal) sensor, the problem of visible (RGB) to Near Infrared (NIR) image fusion has become particularly timely. Indeed, while visible images see color, but suffer from noise, haze, and clouds, the NIR channel captures a clearer picture. The proposed approach fuses these two channels by training a Convolutional Neural Network by Self Supervised Learning (SSL) on a single example. For each such pair, RGB and NIR, the network is trained for seconds to deduce the final fusion. The SSL is based on the comparison of the Structure of Similarity and Edge-Preservation losses, where the labels for the SSL are the input channels themselves. This fusion preserves the relevant detail of each spectral channel without relying on a heavy training process. Experiments demonstrate that the proposed approach achieves similar or better qualitative and quantitative multispectral fusion results than other state-of-the-art methods that do not rely on heavy training and/or large datasets.