Infrared and Visible Image Fusion with ResNet and zero-phase component analysis
This work addresses image fusion for applications like surveillance or medical imaging, but it is incremental as it builds on existing deep learning and feature processing techniques.
The paper tackled the problem of infrared and visible image fusion by proposing a framework that combines ResNet for deep feature extraction with zero-phase component analysis (ZCA) for normalization, resulting in improved performance in objective assessment and visual quality compared to existing methods.
Feature extraction and processing tasks play a key role in Image Fusion, and the fusion performance is directly affected by the different features and processing methods undertaken. By contrast, most of deep learning-based methods use deep features directly without feature extraction or processing. This leads to the fusion performance degradation in some cases. To solve these drawbacks, we propose a deep features and zero-phase component analysis (ZCA) based novel fusion framework is this paper. Firstly, the residual network (ResNet) is used to extract deep features from source images. Then ZCA is utilized to normalize the deep features and obtain initial weight maps. The final weight maps are obtained by employing a soft-max operation in association with the initial weight maps. Finally, the fused image is reconstructed using a weighted-averaging strategy. Compared with the existing fusion methods, experimental results demonstrate that the proposed framework achieves better performance in both objective assessment and visual quality. The code of our fusion algorithm is available at https://github.com/hli1221/imagefusion_resnet50