Emergence of the Primacy Effect in Structured State-Space Models
This reveals a counterintuitive memory bias in SSMs, challenging current theoretical understanding and opening new research avenues for AI memory mechanisms.
The study found that structured state-space models (SSMs), contrary to theoretical expectations of monotonic decay, predominantly retain initially presented data in memory, known as the primacy effect, when tested on a synthetic memorization task.
Structured state-space models (SSMs) have been developed to offer more persistent memory retention than traditional recurrent neural networks, while maintaining real-time inference capabilities and addressing the time-complexity limitations of Transformers. Despite this intended persistence, the memory mechanism of canonical SSMs is theoretically designed to decay monotonically over time, meaning that more recent inputs are expected to be retained more accurately than earlier ones. Contrary to this theoretical expectation, however, the present study reveals a counterintuitive finding: when trained and evaluated on a synthetic, statistically balanced memorization task, SSMs predominantly preserve the *initially* presented data in memory. This pattern of memory bias, known as the *primacy effect* in psychology, presents a non-trivial challenge to the current theoretical understanding of SSMs and opens new avenues for future research.