InvisibleInk: High-Utility and Low-Cost Text Generation with Differential Privacy
This addresses the critical need for scalable and efficient private text generation in applications like retrieval-augmented generation, though it is incremental as it builds on existing differential privacy methods.
The paper tackled the problem of safely incorporating private information into long-form text generation with differential privacy, achieving an 8x reduction in computation cost over state-of-the-art baselines while maintaining utility across privacy levels.
As major progress in LLM-based long-form text generation enables paradigms such as retrieval-augmented generation (RAG) and inference-time scaling, safely incorporating private information into the generation remains a critical open question. We present InvisibleInk, a highly scalable long-form text generation framework satisfying rigorous differential privacy guarantees with respect to the sensitive references. It interprets sampling from the LLM's next-token-distribution as the exponential mechanism over the LLM logits with two innovations. First, we reduce the privacy cost by isolating and clipping only the sensitive information in the model logits (relative to the public logits). Second, we improve text quality by sampling from a small superset of the top-$k$ private tokens. Empirical evaluations demonstrate a consistent $8\times$ reduction in computation cost over state-of-the-art baselines to generate long-form private text of the same utility across privacy levels. In summary, InvisibleInk is able to generate private long-form text at less than $10\times$ the computation cost of non-private generation.