Efficient and Training-Free Control of Language Generation
This addresses the need for efficient and resource-light control of language generation for users of language models, though it is incremental as it builds on existing sampling techniques.
The paper tackles the problem of controllable language generation by proposing Gamma Sampling, a training-free method that incorporates attribute-related information into the sampling process, resulting in improved diversity, attribute relevance, and overall quality when applied to GPT2 compared to baselines.
In recent years, there has been a growing interest in the development of language models capable of generating text with controllable attributes. While several approaches have been proposed, many of these methods require condition-specific data or significant computational resources. In this study, we propose a novel method called Gamma Sampling, which enables controllable language generation without the need for any training data and maintains a fast generation speed. Gamma Sampling incorporates attribute-related information into the sampling process, effectively guiding the language model to produce text with desired attributes. Our experimental results demonstrate that Gamma Sampling, when applied to GPT2, outperforms representative baselines in terms of diversity, attribute relevance, and overall quality of the generated samples.