Memorization or Interpolation ? Detecting LLM Memorization through Input Perturbation Analysis
This addresses concerns about data privacy and reliability in LLMs for researchers and practitioners, though it is incremental as it builds on existing detection methods.
The paper tackles the problem of detecting memorization in Large Language Models (LLMs) by introducing PEARL, a method that uses input perturbation analysis to assess output consistency, and it successfully identified memorization of texts like the Bible and HumanEval code in models such as GPT-4o.
While Large Language Models (LLMs) achieve remarkable performance through training on massive datasets, they can exhibit concerning behaviors such as verbatim reproduction of training data rather than true generalization. This memorization phenomenon raises significant concerns about data privacy, intellectual property rights, and the reliability of model evaluations. This paper introduces PEARL, a novel approach for detecting memorization in LLMs. PEARL assesses how sensitive an LLM's performance is to input perturbations, enabling memorization detection without requiring access to the model's internals. We investigate how input perturbations affect the consistency of outputs, enabling us to distinguish between true generalization and memorization. Our findings, following extensive experiments on the Pythia open model, provide a robust framework for identifying when the model simply regurgitates learned information. Applied on the GPT 4o models, the PEARL framework not only identified cases of memorization of classic texts from the Bible or common code from HumanEval but also demonstrated that it can provide supporting evidence that some data, such as from the New York Times news articles, were likely part of the training data of a given model.