Auditing Language Model Unlearning via Information Decomposition
This work addresses privacy and safety concerns in deploying language models by providing a principled audit for unlearning, though it is incremental as it builds on existing unlearning methods.
The paper exposes a critical limitation in language model unlearning by showing that information about forgotten data remains linearly decodable from internal representations, and it introduces an information-theoretic framework using Partial Information Decomposition to audit this, revealing persistent residual knowledge that correlates with privacy risks.
We expose a critical limitation in current approaches to machine unlearning in language models: despite the apparent success of unlearning algorithms, information about the forgotten data remains linearly decodable from internal representations. To systematically assess this discrepancy, we introduce an interpretable, information-theoretic framework for auditing unlearning using Partial Information Decomposition (PID). By comparing model representations before and after unlearning, we decompose the mutual information with the forgotten data into distinct components, formalizing the notions of unlearned and residual knowledge. Our analysis reveals that redundant information, shared across both models, constitutes residual knowledge that persists post-unlearning and correlates with susceptibility to known adversarial reconstruction attacks. Leveraging these insights, we propose a representation-based risk score that can guide abstention on sensitive inputs at inference time, providing a practical mechanism to mitigate privacy leakage. Our work introduces a principled, representation-level audit for unlearning, offering theoretical insight and actionable tools for safer deployment of language models.