Multiscale Convolutional Transformer with Center Mask Pretraining for Hyperspectral Image Classification
This work addresses the problem of limited labeled data and long-range dependency capture in hyperspectral image classification for remote sensing applications, though it is incremental as it builds on existing transformer and self-supervised techniques.
The authors tackled hyperspectral image classification by proposing a multiscale convolutional transformer with a center mask pretraining method, achieving improved performance on benchmark datasets like Indian Pines and Pavia University with accuracy gains of 2-3% over existing methods.
Hyperspectral images (HSI) not only have a broad macroscopic field of view but also contain rich spectral information, and the types of surface objects can be identified through spectral information, which is one of the main applications in hyperspectral image related research.In recent years, more and more deep learning methods have been proposed, among which convolutional neural networks (CNN) are the most influential. However, CNN-based methods are difficult to capture long-range dependencies, and also require a large amount of labeled data for model training.Besides, most of the self-supervised training methods in the field of HSI classification are based on the reconstruction of input samples, and it is difficult to achieve effective use of unlabeled samples. To address the shortcomings of CNN networks, we propose a noval multi-scale convolutional embedding module for HSI to realize effective extraction of spatial-spectral information, which can be better combined with Transformer network.In order to make more efficient use of unlabeled data, we propose a new self-supervised pretask. Similar to Mask autoencoder, but our pre-training method only masks the corresponding token of the central pixel in the encoder, and inputs the remaining token into the decoder to reconstruct the spectral information of the central pixel.Such a pretask can better model the relationship between the central feature and the domain feature, and obtain more stable training results.