Characterizing Variation in Crowd-Sourced Data for Training Neural Language Generators to Produce Stylistically Varied Outputs
This work addresses the problem of producing stylistically varied outputs in dialogue systems for natural language generation, offering incremental improvements through data analysis and labeling techniques applicable to existing corpora.
The paper tackled the challenge of making neural language generation outputs more natural and varied by analyzing a large crowd-sourced corpus of 50K utterances in the restaurant domain, developing methods to characterize and label stylistic variations, and showing that training with stylistically controlled data or using style labels during training can modify the style of generated utterances while quantifying effects on semantic quality and stylistic control.
One of the biggest challenges of end-to-end language generation from meaning representations in dialogue systems is making the outputs more natural and varied. Here we take a large corpus of 50K crowd-sourced utterances in the restaurant domain and develop text analysis methods that systematically characterize types of sentences in the training data. We then automatically label the training data to allow us to conduct two kinds of experiments with a neural generator. First, we test the effect of training the system with different stylistic partitions and quantify the effect of smaller, but more stylistically controlled training data. Second, we propose a method of labeling the style variants during training, and show that we can modify the style of the generated utterances using our stylistic labels. We contrast and compare these methods that can be used with any existing large corpus, showing how they vary in terms of semantic quality and stylistic control.