Whistledown: Combining User-Level Privacy with Conversational Coherence in LLMs
It addresses privacy concerns for users and enterprises in sensitive conversations with LLMs, but appears incremental as it builds on existing privacy techniques like pseudonymization and differential privacy.
The paper tackles the problem of protecting personally identifiable information in user prompts sent to cloud-hosted LLMs, presenting Whistledown, a privacy layer that combines pseudonymization and ε-local differential privacy with transformation caching to provide best-effort privacy without sacrificing conversational utility, achieving low compute and memory overhead for deployment on client devices or enterprise gateways.
Users increasingly rely on large language models (LLMs) for personal, emotionally charged, and socially sensitive conversations. However, prompts sent to cloud-hosted models can contain personally identifiable information (PII) that users do not want logged, retained, or leaked. We observe this to be especially acute when users discuss friends, coworkers, or adversaries, i.e., when they spill the tea. Enterprises face the same challenge when they want to use LLMs for internal communication and decision-making. In this whitepaper, we present Whistledown, a best-effort privacy layer that modifies prompts before they are sent to the LLM. Whistledown combines pseudonymization and $ε$-local differential privacy ($ε$-LDP) with transformation caching to provide best-effort privacy protection without sacrificing conversational utility. Whistledown is designed to have low compute and memory overhead, allowing it to be deployed directly on a client's device in the case of individual users. For enterprise users, Whistledown is deployed centrally within a zero-trust gateway that runs on an enterprise's trusted infrastructure. Whistledown requires no changes to the existing APIs of popular LLM providers.