Simulating Human Memory with Language Models
For researchers building user simulators, this work identifies a gap in memory fidelity and offers methods to bridge it, though results are preliminary.
Language models exhibit better memory than humans in classic psychology experiments, but prompting strategies and a compactor can induce more human-like forgetting, leading to more effective user simulators in an education task.
Language models are increasingly being deployed as user simulators, but their memory is far more reliable than that of real users. To measure this gap, we run a series of classic memory experiments from psychology on both humans and language models. Across tasks, we find that out-of-the-box language models exhibit better memory than humans, even when prompted to imitate human behavior. We then show that better prompting strategies and the use of a compactor can cause language models to forget content in a more human-like way. Using these methods, we show preliminary evidence that language models with human-like memory constraints can function as more effective user simulators in a downstream education task. Finally, we release human reference data and benchmarks to support future work on simulating human memory with language models.