DistRAG: Towards Distance-Based Spatial Reasoning in LLMs
This addresses spatial reasoning for tasks like POI recommendation and itinerary planning, offering a flexible step towards a world model for LLMs, but it is incremental as it builds on existing retrieval techniques.
The paper tackles the problem of LLMs lacking reliable spatial reasoning capabilities, especially for distances, by developing DistRAG, a method that retrieves relevant spatial information from a graph to enable LLMs to answer distance-based reasoning questions they otherwise cannot.
Many real world tasks where Large Language Models (LLMs) can be used require spatial reasoning, like Point of Interest (POI) recommendation and itinerary planning. However, on their own LLMs lack reliable spatial reasoning capabilities, especially about distances. To address this problem, we develop a novel approach, DistRAG, that enables an LLM to retrieve relevant spatial information not explicitly learned during training. Our method encodes the geodesic distances between cities and towns in a graph and retrieves a context subgraph relevant to the question. Using this technique, our method enables an LLM to answer distance-based reasoning questions that it otherwise cannot answer. Given the vast array of possible places an LLM could be asked about, DistRAG offers a flexible first step towards providing a rudimentary `world model' to complement the linguistic knowledge held in LLMs.