Understanding the Geospatial Reasoning Capabilities of LLMs: A Trajectory Recovery Perspective
This work addresses the challenge of assessing LLMs' ability to understand and navigate road networks, which is incremental in applying existing models to a new domain-specific task.
The paper tackles the problem of evaluating geospatial reasoning in LLMs by using trajectory recovery as a proxy task, showing that LLMs outperform specialized models and baselines with strong zero-shot generalization on a dataset of over 4,000 real-world trajectories.
We explore the geospatial reasoning capabilities of Large Language Models (LLMs), specifically, whether LLMs can read road network maps and perform navigation. We frame trajectory recovery as a proxy task, which requires models to reconstruct masked GPS traces, and introduce GLOBALTRACE, a dataset with over 4,000 real-world trajectories across diverse regions and transportation modes. Using road network as context, our prompting framework enables LLMs to generate valid paths without accessing any external navigation tools. Experiments show that LLMs outperform off-the-shelf baselines and specialized trajectory recovery models, with strong zero-shot generalization. Fine-grained analysis shows that LLMs have strong comprehension of the road network and coordinate systems, but also pose systematic biases with respect to regions and transportation modes. Finally, we demonstrate how LLMs can enhance navigation experiences by reasoning over maps in flexible ways to incorporate user preferences.