Constrained Text Generation with Global Guidance -- Case Study on CommonGen
This work addresses the problem of generating coherent and comprehensive text under constraints for natural language processing applications, representing an incremental advance by applying reinforcement learning to a known bottleneck in traditional methods.
The paper tackled constrained text generation, specifically the CommonGen task of generating text from concept sets, by using reinforcement learning to incorporate global constraints like fluency, common sense, and concept coverage as rewards, resulting in significant improvements in concept coverage and outperforming existing models in automatic evaluations.
This paper studies constrained text generation, which is to generate sentences under certain pre-conditions. We focus on CommonGen, the task of generating text based on a set of concepts, as a representative task of constrained text generation. Traditional methods mainly rely on supervised training to maximize the likelihood of target sentences.However, global constraints such as common sense and coverage cannot be incorporated into the likelihood objective of the autoregressive decoding process. In this paper, we consider using reinforcement learning to address the limitation, measuring global constraints including fluency, common sense and concept coverage with a comprehensive score, which serves as the reward for reinforcement learning. Besides, we design a guided decoding method at the word, fragment and sentence levels. Experiments demonstrate that our method significantly increases the concept coverage and outperforms existing models in various automatic evaluations.