Privacy-Preserving In-Context Learning for Large Language Models
This addresses privacy risks for users of LLMs in ICL applications, but it is incremental as it applies differential privacy to a known bottleneck.
The paper tackles the problem of sensitive private information leakage in large language models' in-context learning responses by proposing Differentially Private In-context Learning (DP-ICL), which achieves a strong utility-privacy tradeoff as shown in evaluations on text classification and language generation tasks.
In-context learning (ICL) is an important capability of Large Language Models (LLMs), enabling these models to dynamically adapt based on specific, in-context exemplars, thereby improving accuracy and relevance. However, LLM's responses may leak the sensitive private information contained in in-context exemplars. To address this challenge, we propose Differentially Private In-context Learning (DP-ICL), a general paradigm for privatizing ICL tasks. The key idea for DP-ICL paradigm is generating differentially private responses through a noisy consensus among an ensemble of LLM's responses based on disjoint exemplar sets. Based on the general paradigm of DP-ICL, we instantiate several techniques showing how to privatize ICL for text classification and language generation. We evaluate DP-ICL on four text classification benchmarks and two language generation tasks, and our empirical results show that DP-ICL achieves a strong utility-privacy tradeoff.