Memory, Space, and Planning: Multiscale Predictive Representations
This is an incremental review that synthesizes existing evidence to advance understanding of brain mechanisms and inform AI development.
The paper tackles the problem of understanding how memory, prediction, and planning are integrated in biological and artificial agents, arguing that they rely on multiscale predictive representations in brain hierarchies, which enhances recall and generalization.
Memory is inherently entangled with prediction and planning. Flexible behavior in biological and artificial agents depends on the interplay of learning from the past and predicting the future in ever-changing environments. This chapter reviews computational, behavioral, and neural evidence suggesting these processes rely on learning the relational structure of experiences, known as cognitive maps, and draws two key takeaways. First, that these memory structures are organized as multiscale, compact predictive representations in hippocampal and prefrontal cortex, or PFC, hierarchies. Second, we argue that such predictive memory structures are crucial to the complementary functions of the hippocampus and PFC, both for enabling a recall of detailed and coherent past episodes as well as generalizing experiences at varying scales for efficient prediction and planning. These insights advance our understanding of memory and planning mechanisms in the brain and hold significant implications for advancing artificial intelligence systems.