PREADD: Prefix-Adaptive Decoding for Controlled Text Generation
This addresses the problem of generating text with specific attributes for users needing flexible control, offering a novel method that is incremental in improving over existing prompting and expert-based approaches.
The authors tackled controlled text generation by proposing Prefix-Adaptive Decoding (PREADD), which linearly combines output logits from multiple prompts without external models, and found it outperformed baselines by 12% or more in relative gain on tasks like toxic output mitigation, gender bias reduction, and sentiment control.
We propose Prefix-Adaptive Decoding (PREADD), a flexible method for controlled text generation. Unlike existing methods that use auxiliary expert models to control for attributes, PREADD does not require an external model, instead relying on linearly combining output logits from multiple prompts. Specifically, PREADD contrasts the output logits generated using a raw prompt against those generated using a prefix-prepended prompt, enabling both positive and negative control with respect to any attribute encapsulated by the prefix. We evaluate PREADD on three tasks -- toxic output mitigation, gender bias reduction, and sentiment control -- and find that PREADD outperforms not only prompting baselines, but also an auxiliary-expert control method, by 12% or more in relative gain on our main metrics for each task.