Transferring Physical Priors into Remote Sensing Segmentation via Large Language Models
It addresses the problem of integrating diverse physical variables into remote sensing segmentation for Earth observation, offering a flexible approach that avoids costly retraining for new sensors.
This paper introduces a novel paradigm for integrating physical priors (e.g., DEM, SAR, NDVI) into remote sensing segmentation without retraining foundation models. The proposed PriorSeg model improves segmentation accuracy and physical plausibility, achieving state-of-the-art results on heterogeneous settings.
Semantic segmentation of remote sensing imagery is fundamental to Earth observation. Achieving accurate results requires integrating not only optical images but also physical variables such as the Digital Elevation Model (DEM), Synthetic Aperture Radar (SAR) and Normalized Difference Vegetation Index (NDVI). Recent foundation models (FMs) leverage pre-training to exploit these variables but still depend on spatially aligned data and costly retraining when involving new sensors. To overcome these limitations, we introduce a novel paradigm for integrating domain-specific physical priors into segmentation models. We first construct a Physical-Centric Knowledge Graph (PCKG) by prompting large language models to extract physical priors from 1,763 vocabularies, and use it to build a heterogeneous, spatial-aligned dataset, Phy-Sky-SA. Building on this foundation, we develop PriorSeg, a physics-aware residual refinement model trained with a joint visual-physical strategy that incorporates a novel physics-consistency loss. Experiments on heterogeneous settings demonstrate that PriorSeg improves segmentation accuracy and physical plausibility without retraining the FMs. Ablation studies verify the effectiveness of the Phy-Sky-SA dataset, the PCKG, and the physics-consistency loss.