Timesteps of Mamba Align with Human Reading Times
For cognitive scientists, it provides a new computational model that captures real-time language processing with continuous memory, offering insights beyond transformer-based models.
This paper shows that per-word timesteps from the Mamba state-space model significantly predict human reading times, even when controlling for GPT-2 surprisal, suggesting Mamba's dynamic memory aligns with human processing.
This study demonstrates an alignment of per-word processing time in a popular state-space language model Mamba and human readers. In Mamba, the recurrent state transition at each layer conceptually takes some duration of time, the discretization timestep $Δ_t$, determined dynamically in response to the input. Using a naturalistic reading dataset, we show that the per-word timestep from Mamba is a significant predictor of human reading times, and remains significant even when known predictors such as GPT-2 surprisal are controlled for. We further suggest, through formal analysis of Mamba's architecture and internal dynamics, that Mamba can serve as a new, valuable lens to look at human real-time language processing with ever-updated memory, because it allows us to look at how each module (layer) weighs short- and long-term information retention, and how noise may interact with dynamic, continuous memory representation. Code is available online.