SatBLIP: Context Understanding and Feature Identification from Satellite Imagery with Vision-Language Learning
This work addresses the need for fine-grained, interpretable vulnerability assessment in rural areas, where standard indices are coarse and prior remote sensing methods rely on handcrafted features or manual audits.
SatBLIP predicts county-level Social Vulnerability Index (SVI) from satellite imagery by combining contrastive image-text alignment with bootstrapped captioning tailored to satellite semantics, achieving interpretable mapping of rural risk environments.
Rural environmental risks are shaped by place-based conditions (e.g., housing quality, road access, land-surface patterns), yet standard vulnerability indices are coarse and provide limited insight into risk contexts. We propose SatBLIP, a satellite-specific vision-language framework for rural context understanding and feature identification that predicts county-level Social Vulnerability Index (SVI). SatBLIP addresses limitations of prior remote sensing pipelines-handcrafted features, manual virtual audits, and natural-image-trained VLMs-by coupling contrastive image-text alignment with bootstrapped captioning tailored to satellite semantics. We use GPT-4o to generate structured descriptions of satellite tiles (roof type/condition, house size, yard attributes, greenery, and road context), then fine-tune a satellite-adapted BLIP model to generate captions for unseen images. Captions are encoded with CLIP and fused with LLM-derived embeddings via attention for SVI estimation under spatial aggregation. Using SHAP, we identify salient attributes (e.g., roof form/condition, street width, vegetation, cars/open space) that consistently drive robust predictions, enabling interpretable mapping of rural risk environments.