A Distributional Approach to Controlled Text Generation
This work addresses the problem of controlling text generation from pre-trained language models for researchers and practitioners, offering a more general framework for specifying diverse constraints.
This paper introduces a Distributional Approach for Controlled Text Generation using pre-trained Language Models, allowing the specification of both pointwise and distributional constraints within a single framework. The method minimizes KL divergence from the initial LM and trains a controlled Autoregressive LM using an adaptive distributional Policy Gradient. Experiments show advantages over baselines in balancing constraint satisfaction with divergence for pointwise constraints and demonstrate potential for mitigating bias with distributional constraints.
We propose a Distributional Approach for addressing Controlled Text Generation from pre-trained Language Models (LMs). This approach permits to specify, in a single formal framework, both "pointwise" and "distributional" constraints over the target LM -- to our knowledge, the first model with such generality -- while minimizing KL divergence from the initial LM distribution. The optimal target distribution is then uniquely determined as an explicit EBM (Energy-Based Model) representation. From that optimal representation we then train a target controlled Autoregressive LM through an adaptive distributional variant of Policy Gradient. We conduct a first set of experiments over pointwise constraints showing the advantages of our approach over a set of baselines, in terms of obtaining a controlled LM balancing constraint satisfaction with divergence from the initial LM. We then perform experiments over distributional constraints, a unique feature of our approach, demonstrating its potential as a remedy to the problem of Bias in Language Models. Through an ablation study, we show the effectiveness of our adaptive technique for obtaining faster convergence. (Code available at https://github.com/naver/gdc)