An Interactively Reinforced Paradigm for Joint Infrared-Visible Image Fusion and Saliency Object Detection
This work addresses the problem of discovering and localizing hidden objects in the wild for unmanned systems, representing an incremental advancement by integrating two existing tasks into a novel collaborative framework.
The paper tackles the joint problem of infrared-visible image fusion and salient object detection by proposing an interactively reinforced multi-task paradigm (IRFS) that uses a feature screening fusion subnetwork and a fusion-guided SOD subnetwork to enhance both tasks, achieving improved performance with fewer parameters and shorter training times.
This research focuses on the discovery and localization of hidden objects in the wild and serves unmanned systems. Through empirical analysis, infrared and visible image fusion (IVIF) enables hard-to-find objects apparent, whereas multimodal salient object detection (SOD) accurately delineates the precise spatial location of objects within the picture. Their common characteristic of seeking complementary cues from different source images motivates us to explore the collaborative relationship between Fusion and Salient object detection tasks on infrared and visible images via an Interactively Reinforced multi-task paradigm for the first time, termed IRFS. To the seamless bridge of multimodal image fusion and SOD tasks, we specifically develop a Feature Screening-based Fusion subnetwork (FSFNet) to screen out interfering features from source images, thereby preserving saliency-related features. After generating the fused image through FSFNet, it is then fed into the subsequent Fusion-Guided Cross-Complementary SOD subnetwork (FC$^2$Net) as the third modality to drive the precise prediction of the saliency map by leveraging the complementary information derived from the fused image. In addition, we develop an interactive loop learning strategy to achieve the mutual reinforcement of IVIF and SOD tasks with a shorter training period and fewer network parameters. Comprehensive experiment results demonstrate that the seamless bridge of IVIF and SOD mutually enhances their performance, and highlights their superiority.