Harry Potter is Still Here! Probing Knowledge Leakage in Targeted Unlearned Large Language Models via Automated Adversarial Prompting
This addresses a critical limitation in evaluating unlearning algorithms for LLMs, which is important for researchers and practitioners in AI safety and model robustness.
The authors tackled the problem of hidden retained knowledge in unlearned large language models by developing LURK, a framework that uses adversarial suffix prompting to probe for residual information in the Harry Potter domain, revealing that even successfully unlearned models can leak specific details under targeted attacks.
This work presents LURK (Latent UnleaRned Knowledge), a novel framework that probes for hidden retained knowledge in unlearned LLMs through adversarial suffix prompting. LURK automatically generates adversarial prompt suffixes designed to elicit residual knowledge about the Harry Potter domain, a commonly used benchmark for unlearning. Our experiments reveal that even models deemed successfully unlearned can leak idiosyncratic information under targeted adversarial conditions, highlighting critical limitations of current unlearning evaluation standards. By uncovering latent knowledge through indirect probing, LURK offers a more rigorous and diagnostic tool for assessing the robustness of unlearning algorithms. All code will be publicly available.