Meta-Memory: Retrieving and Integrating Semantic-Spatial Memories for Robot Spatial Reasoning
It addresses the underexplored challenge of efficient memory retrieval and integration for robot spatial reasoning, with practical deployment in complex environments.
The paper tackles the problem of enabling robots to answer human queries about spatial locations by proposing Meta-Memory, an LLM-driven agent that retrieves and integrates semantic-spatial memories, which significantly outperforms state-of-the-art methods on benchmarks like SpaceLocQA and NaVQA.
Navigating complex environments requires robots to effectively store observations as memories and leverage them to answer human queries about spatial locations, which is a critical yet underexplored research challenge. While prior work has made progress in constructing robotic memory, few have addressed the principled mechanisms needed for efficient memory retrieval and integration. To bridge this gap, we propose Meta-Memory, a large language model (LLM)-driven agent that constructs a high-density memory representation of the environment. The key innovation of Meta-Memory lies in its capacity to retrieve and integrate relevant memories through joint reasoning over semantic and spatial modalities in response to natural language location queries, thereby empowering robots with robust and accurate spatial reasoning capabilities. To evaluate its performance, we introduce SpaceLocQA, a large-scale dataset encompassing diverse real-world spatial question-answering scenarios. Experimental results show that Meta-Memory significantly outperforms state-of-the-art methods on both the SpaceLocQA and the public NaVQA benchmarks. Furthermore, we successfully deployed Meta-Memory on real-world robotic platforms, demonstrating its practical utility in complex environments. Project page: https://itsbaymax.github.io/meta-memory.github.io/ .