Spectral Gaps and Spatial Priors: Studying Hyperspectral Downstream Adaptation Using TerraMind
This work addresses the challenge of integrating hyperspectral data into multimodal models for geospatial applications, but it is incremental as it establishes a baseline rather than introducing a new method.
The study tackled the problem of adapting a geospatial foundation model (TerraMind) to hyperspectral imaging downstream tasks without specific pretraining, finding that deep learning models with native hyperspectral support generally outperform it, though TerraMind can adapt with moderate performance decline.
Geospatial Foundation Models (GFMs) typically lack native support for Hyperspectral Imaging (HSI) due to the complexity and sheer size of high-dimensional spectral data. This study investigates the adaptability of TerraMind, a multimodal GFM, to address HSI downstream tasks \emph{without} HSI-specific pretraining. Therefore, we implement and compare two channel adaptation strategies: Naive Band Selection and physics-aware Spectral Response Function (SRF) grouping. Overall, our results indicate a general superiority of deep learning models with native support of HSI data. Our experiments also demonstrate the ability of TerraMind to adapt to HSI downstream tasks through band selection with moderate performance decline. Therefore, the findings of this research establish a critical baseline for HSI integration, motivating the need for native spectral tokenization in future multimodal model architectures.