Anti-adversarial Learning: Desensitizing Prompts for Large Language Models
This addresses privacy risks for users of cloud-based LLMs, offering a practical solution compared to traditional methods, though it is incremental as it builds on existing techniques like masked language modeling.
The paper tackles the problem of preserving privacy in user prompts for large language models by proposing PromptObfus, a method that desensitizes prompts to obscure sensitive information while maintaining model prediction stability, achieving effective privacy prevention with minimal performance disruption on three NLP tasks.
With the widespread use of LLMs, preserving privacy in user prompts has become crucial, as prompts risk exposing privacy and sensitive data to the cloud LLMs. Traditional techniques like homomorphic encryption, secure multi-party computation, and federated learning face challenges due to heavy computational costs and user participation requirements, limiting their applicability in LLM scenarios. In this paper, we propose PromptObfus, a novel method for desensitizing LLM prompts. The core idea of PromptObfus is "anti-adversarial" learning, which perturbs privacy words in the prompt to obscure sensitive information while retaining the stability of model predictions. Specifically, PromptObfus frames prompt desensitization as a masked language modeling task, replacing privacy-sensitive terms with a [MASK] token. A desensitization model is trained to generate candidate replacements for each masked position. These candidates are subsequently selected based on gradient feedback from a surrogate model, ensuring minimal disruption to the task output. We demonstrate the effectiveness of our approach on three NLP tasks. Results show that PromptObfus effectively prevents privacy inference from remote LLMs while preserving task performance.