Transformers Remember First, Forget Last: Dual-Process Interference in LLMs
This reveals a primacy bias in transformers that could impact interference-heavy applications like question-answering or summarization, though it is incremental in applying cognitive psychology paradigms to LLMs.
The study investigated how large language models handle conflicting information in context, finding that they universally prioritize early memories over recent ones, with proactive interference dominating retroactive interference (Cohen's d = 1.73, p < 0.0001), opposite to human memory patterns.
When large language models encounter conflicting information in context, which memories survive -- early or recent? We adapt classical interference paradigms from cognitive psychology to answer this question, testing 39 LLMs across diverse architectures and scales. Every model shows the same pattern: proactive interference (PI) dominates retroactive interference (RI) universally (Cohen's d = 1.73, p < 0.0001), meaning early encodings are protected at the cost of recent information -- the opposite of human memory, where RI typically dominates. Three findings indicate that RI and PI reflect separate memory mechanisms. RI and PI are uncorrelated (R^2 = 0.044), rejecting a unified "memory capacity." Model size predicts RI resistance (R^2 = 0.49) but not PI (R^2 = 0.06, n.s.) -- only RI is capacity-dependent. And error analysis reveals distinct failure modes: RI failures are passive retrieval failures (51%), while PI failures show active primacy intrusion (56%); both show <1% hallucination. These patterns parallel the consolidation-retrieval distinction in cognitive science, suggesting that transformer attention creates a primacy bias with direct implications for interference-heavy applications.