OmniGeo: Towards a Multimodal Large Language Models for Geospatial Artificial Intelligence
This work addresses the problem of integrating multimodal data for geospatial AI applications, which is incremental as it adapts existing MLLM frameworks to a specific domain.
The paper tackles the challenge of applying multimodal large language models to geospatial artificial intelligence by proposing OmniGeo, a model that processes heterogeneous data like satellite imagery and text, resulting in outperforming task-specific models and existing LLMs on diverse geospatial tasks with competitive zero-shot performance.
The rapid advancement of multimodal large language models (LLMs) has opened new frontiers in artificial intelligence, enabling the integration of diverse large-scale data types such as text, images, and spatial information. In this paper, we explore the potential of multimodal LLMs (MLLM) for geospatial artificial intelligence (GeoAI), a field that leverages spatial data to address challenges in domains including Geospatial Semantics, Health Geography, Urban Geography, Urban Perception, and Remote Sensing. We propose a MLLM (OmniGeo) tailored to geospatial applications, capable of processing and analyzing heterogeneous data sources, including satellite imagery, geospatial metadata, and textual descriptions. By combining the strengths of natural language understanding and spatial reasoning, our model enhances the ability of instruction following and the accuracy of GeoAI systems. Results demonstrate that our model outperforms task-specific models and existing LLMs on diverse geospatial tasks, effectively addressing the multimodality nature while achieving competitive results on the zero-shot geospatial tasks. Our code will be released after publication.