Beyond Frequency: The Role of Redundancy in Large Language Model Memorization
This addresses privacy and fairness risks in large language models, offering insights for data preprocessing to mitigate these issues, though it is incremental as it builds on prior work on memorization factors.
The study tackled the problem of memorization in large language models by analyzing the role of redundancy, revealing that low-redundancy samples are more vulnerable to memorization, with 79% of memorized samples being low-redundancy and showing 2-fold higher vulnerability than high-redundancy ones.
Memorization in large language models poses critical risks for privacy and fairness as these systems scale to billions of parameters. While previous studies established correlations between memorization and factors like token frequency and repetition patterns, we revealed distinct response patterns: frequency increases minimally impact memorized samples (e.g. 0.09) while substantially affecting non-memorized samples (e.g., 0.25), with consistency observed across model scales. Through counterfactual analysis by perturbing sample prefixes and quantifying perturbation strength through token positional changes, we demonstrate that redundancy correlates with memorization patterns. Our findings establish that: about 79% of memorized samples are low-redundancy, these low-redundancy samples exhibit 2-fold higher vulnerability than high-redundancy ones, and consequently memorized samples drop by 0.6 under perturbation while non-memorized samples drop by only 0.01, indicating that more redundant content becomes both more memorable and more fragile. These findings suggest potential redundancy-guided approaches for data preprocessing, thereby reducing privacy risks and mitigating bias to ensure fairness in model deployments.