Leaking LoRa: An Evaluation of Password Leaks and Knowledge Storage in Large Language Models
This addresses a security problem for users and developers of LLMs by highlighting vulnerabilities in fine-tuning processes, though it is incremental as it builds on existing methods for model editing.
The study investigated the risk of password leakage when fine-tuning Large Language Models on user data containing sensitive information, successfully recovering 37 out of 200 passwords from a fine-tuned model and reducing this to 0 after applying a removal technique.
To effectively deploy Large Language Models (LLMs) in application-specific settings, fine-tuning techniques are applied to enhance performance on specialized tasks. This process often involves fine-tuning on user data data, which may contain sensitive information. Although not recommended, it is not uncommon for users to send passwords in messages, and fine-tuning models on this could result in passwords being leaked. In this study, a Large Language Model is fine-tuned with customer support data and passwords from the RockYou password wordlist using Low-Rank Adaptation (LoRA). Out of the first 200 passwords from the list, 37 were successfully recovered. Further, causal tracing is used to identify that password information is largely located in a few layers. Lastly, Rank One Model Editing (ROME) is used to remove the password information from the model, resulting in the number of passwords recovered going from 37 to 0.