Gradient-Based Constrained Sampling from Language Models
This addresses the challenge of controllably sampling from language models for users needing specific text outputs, though it is incremental as it builds on existing methods.
The paper tackled the problem of constrained sampling from large language models to generate text that satisfies user-defined constraints while maintaining fluency and downstream task performance, achieving significant improvements over competitive baselines in toxicity avoidance, sentiment control, and keyword-guided generation.
Large pretrained language models generate fluent text but are notoriously hard to controllably sample from. In this work, we study constrained sampling from such language models: generating text that satisfies user-defined constraints, while maintaining fluency and the model's performance in a downstream task. We propose MuCoLa -- a sampling procedure that combines the log-likelihood of the language model with arbitrary (differentiable) constraints in a single energy function, and then generates samples in a non-autoregressive manner. Specifically, it initializes the entire output sequence with noise and follows a Markov chain defined by Langevin Dynamics using the gradients of the energy function. We evaluate MuCoLa on text generation with soft and hard constraints as well as their combinations obtaining significant improvements over competitive baselines for toxicity avoidance, sentiment control, and keyword-guided generation.