Quick Starting Dialog Systems with Paraphrase Generation
This work addresses the challenge of quickly deploying dialog systems at scale within organizations, though it is incremental as it applies existing neural paraphrasing methods to new data.
The paper tackled the problem of reducing the cost and effort in creating conversational agents by using paraphrase generation to artificially generate more training data from existing examples, resulting in increased generalization capabilities for intent classification models on English and French datasets.
Acquiring training data to improve the robustness of dialog systems can be a painstakingly long process. In this work, we propose a method to reduce the cost and effort of creating new conversational agents by artificially generating more data from existing examples, using paraphrase generation. Our proposed approach can kick-start a dialog system with little human effort, and brings its performance to a level satisfactory enough for allowing actual interactions with real end-users. We experimented with two neural paraphrasing approaches, namely Neural Machine Translation and a Transformer-based seq2seq model. We present the results obtained with two datasets in English and in French:~a crowd-sourced public intent classification dataset and our own corporate dialog system dataset. We show that our proposed approach increased the generalization capabilities of the intent classification model on both datasets, reducing the effort required to initialize a new dialog system and helping to deploy this technology at scale within an organization.