Hyperspectral Image Super-Resolution with Spectral Mixup and Heterogeneous Datasets
This work solves the problem of limited training data for hyperspectral image super-resolution, which is important for remote sensing and imaging applications, but it is incremental as it builds on existing neural network approaches.
The paper tackles hyperspectral image super-resolution by addressing data limitations and network issues with a multi-tasking network, spectral mixup augmentation, and semi-supervised learning, achieving significant performance improvements over existing methods on four standard datasets.
This work studies Hyperspectral image (HSI) super-resolution (SR). HSI SR is characterized by high-dimensional data and a limited amount of training examples. This exacerbates the undesirable behaviors of neural networks such as memorization and sensitivity to out-of-distribution samples. This work addresses these issues with three contributions. First, we observe that HSI SR and RGB image SR are correlated and develop a novel multi-tasking network to train them jointly so that the auxiliary task RGB image SR can provide additional supervision. Second, we propose a simple, yet effective data augmentation routine, termed Spectral Mixup, to construct effective virtual training samples to enlarge the training set. Finally, we extend the network to a semi-supervised setting so that it can learn from datasets containing only low-resolution HSIs. With these contributions, our method is able to learn from heterogeneous datasets and lift the requirement for having a large amount of HD HSI training samples. Extensive experiments on four standard datasets show that our method outperforms existing methods significantly and underpin the relevance of our contributions. Code has been made available at https://github.com/kli8996/HSISR.