Cooperative Colorization: Exploring Latent Cross-Domain Priors for NIR Image Spectrum Translation
This addresses a domain-specific problem in computer vision for applications like remote sensing or medical imaging, but it is incremental as it builds on existing colorization and translation methods.
The paper tackles the problem of near-infrared (NIR) image spectrum translation, which is challenging due to mapping ambiguity and limited training data, by proposing a cooperative learning framework called CoColor that explores latent cross-domain priors. The result is improved performance, outperforming state-of-the-art methods by 3.95dB and 4.66dB in PSNR for NIR and grayscale colorization tasks, respectively.
Near-infrared (NIR) image spectrum translation is a challenging problem with many promising applications. Existing methods struggle with the mapping ambiguity between the NIR and the RGB domains, and generalize poorly due to the limitations of models' learning capabilities and the unavailability of sufficient NIR-RGB image pairs for training. To address these challenges, we propose a cooperative learning paradigm that colorizes NIR images in parallel with another proxy grayscale colorization task by exploring latent cross-domain priors (i.e., latent spectrum context priors and task domain priors), dubbed CoColor. The complementary statistical and semantic spectrum information from these two task domains -- in the forms of pre-trained colorization networks -- are brought in as task domain priors. A bilateral domain translation module is subsequently designed, in which intermittent NIR images are generated from grayscale and colorized in parallel with authentic NIR images; and vice versa for the grayscale images. These intermittent transformations act as latent spectrum context priors for efficient domain knowledge exchange. We progressively fine-tune and fuse these modules with a series of pixel-level and feature-level consistency constraints. Experiments show that our proposed cooperative learning framework produces satisfactory spectrum translation outputs with diverse colors and rich textures, and outperforms state-of-the-art counterparts by 3.95dB and 4.66dB in terms of PNSR for the NIR and grayscale colorization tasks, respectively.