Retracing the Past: LLMs Emit Training Data When They Get Lost
This addresses privacy and copyright concerns for LLM users by providing a more systematic method to assess memorization vulnerabilities, though it is incremental in improving upon existing extraction techniques.
The paper tackled the problem of extracting memorized training data from large language models (LLMs) by introducing Confusion-Inducing Attacks (CIA), a framework that systematically maximizes model uncertainty to induce data leakage, and demonstrated that it outperforms existing baselines in extracting verbatim and near-verbatim data without prior knowledge.
The memorization of training data in large language models (LLMs) poses significant privacy and copyright concerns. Existing data extraction methods, particularly heuristic-based divergence attacks, often exhibit limited success and offer limited insight into the fundamental drivers of memorization leakage. This paper introduces Confusion-Inducing Attacks (CIA), a principled framework for extracting memorized data by systematically maximizing model uncertainty. We empirically demonstrate that the emission of memorized text during divergence is preceded by a sustained spike in token-level prediction entropy. CIA leverages this insight by optimizing input snippets to deliberately induce this consecutive high-entropy state. For aligned LLMs, we further propose Mismatched Supervised Fine-tuning (SFT) to simultaneously weaken their alignment and induce targeted confusion, thereby increasing susceptibility to our attacks. Experiments on various unaligned and aligned LLMs demonstrate that our proposed attacks outperform existing baselines in extracting verbatim and near-verbatim training data without requiring prior knowledge of the training data. Our findings highlight persistent memorization risks across various LLMs and offer a more systematic method for assessing these vulnerabilities.