The Mosaic Memory of Large Language Models
This work addresses the issue of data memorization in LLMs for researchers and practitioners, showing it is more complex than previously thought, with incremental insights into its syntactic nature and real-world prevalence.
The study tackled the problem of how Large Language Models (LLMs) memorize training data, revealing that memorization occurs through assembling information from similar sequences, termed mosaic memory, with fuzzy duplicates contributing up to 0.8 as much as exact duplicates. This challenges the assumption that memorization only results from repeated sequences and has implications for privacy and model evaluation.
As Large Language Models (LLMs) become widely adopted, understanding how they learn from, and memorize, training data becomes crucial. Memorization in LLMs is widely assumed to only occur as a result of sequences being repeated in the training data. Instead, we show that LLMs memorize by assembling information from similar sequences, a phenomena we call mosaic memory. We show major LLMs to exhibit mosaic memory, with fuzzy duplicates contributing to memorization as much as 0.8 of an exact duplicate and even heavily modified sequences contributing substantially to memorization. Despite models display reasoning capabilities, we somewhat surprisingly show memorization to be predominantly syntactic rather than semantic. We finally show fuzzy duplicates to be ubiquitous in real-world data, untouched by deduplication techniques. Taken together, our results challenge widely held beliefs and show memorization to be a more complex, mosaic process, with real-world implications for privacy, confidentiality, model utility and evaluation.