A Fictional Q&A Dataset for Studying Memorization and Knowledge Acquisition
This work provides a tool for researchers studying memorization in language models, but it is incremental as it builds on existing datasets and methods.
The authors tackled the problem of understanding how language models memorize facts versus verbatim sequences by creating a new synthetic Q&A dataset about fictional events, enabling experiments that tease apart these memorization forms.
When language models are trained on textual data, they acquire both knowledge about the structure of language as well as knowledge of facts about the world. At inference time, their knowledge of facts can be leveraged to solve interesting problems and perform useful knowledge work for users. It is well known that language models can verbatim memorize long sequences from their training data. However, it is much less well understood how language models memorize facts seen during training. In this work, we propose a new dataset to specifically empower researchers to study the dual processes of fact memorization and verbatim sequence memorization. The dataset consists of synthetically-generated, webtext-like documents about fictional events, as well as question-answer pairs about the events. We conduct training experiments showing how synthetic data about fictional events can be effective in teasing apart different forms of memorization. We also document the challenges in effectively building realistic, fictional synthetic data.