Vector Quantization in the Brain: Grid-like Codes in World Models
This work provides a computational tool for efficient sequence modeling and a theoretical perspective on grid-like codes in neural systems, but it appears incremental as it builds on existing vector quantization and brain-inspired approaches.
The authors tackled the problem of compressing observation-action sequences by proposing Grid-like Code Quantization (GCQ), a brain-inspired method that uses grid-like patterns and action-conditioned codebooks to achieve spatiotemporal compression, resulting in a unified world model that supports long-horizon prediction and planning.
We propose Grid-like Code Quantization (GCQ), a brain-inspired method for compressing observation-action sequences into discrete representations using grid-like patterns in attractor dynamics. Unlike conventional vector quantization approaches that operate on static inputs, GCQ performs spatiotemporal compression through an action-conditioned codebook, where codewords are derived from continuous attractor neural networks and dynamically selected based on actions. This enables GCQ to jointly compress space and time, serving as a unified world model. The resulting representation supports long-horizon prediction, goal-directed planning, and inverse modeling. Experiments across diverse tasks demonstrate GCQ's effectiveness in compact encoding and downstream performance. Our work offers both a computational tool for efficient sequence modeling and a theoretical perspective on the formation of grid-like codes in neural systems.