Last One Standing: A Comparative Analysis of Security and Privacy of Soft Prompt Tuning, LoRA, and In-Context Learning
This work addresses privacy and security challenges for users adapting LLMs with private data, but it is incremental as it systematically evaluates existing techniques without introducing new methods.
The paper tackled the problem of comparing the privacy and security of three LLM adaptation techniques—Soft Prompt Tuning, LoRA, and In-Context Learning—against attacks like membership inference, backdoor, and model stealing, finding that no single technique is universally secure and each has distinct vulnerabilities.
Large Language Models (LLMs) are powerful tools for natural language processing, enabling novel applications and user experiences. However, to achieve optimal performance, LLMs often require adaptation with private data, which poses privacy and security challenges. Several techniques have been proposed to adapt LLMs with private data, such as Low-Rank Adaptation (LoRA), Soft Prompt Tuning (SPT), and In-Context Learning (ICL), but their comparative privacy and security properties have not been systematically investigated. In this work, we fill this gap by evaluating the robustness of LoRA, SPT, and ICL against three types of well-established attacks: membership inference, which exposes data leakage (privacy); backdoor, which injects malicious behavior (security); and model stealing, which can violate intellectual property (privacy and security). Our results show that there is no silver bullet for privacy and security in LLM adaptation and each technique has different strengths and weaknesses.