Simulated Chats for Building Dialog Systems: Learning to Generate Conversations from Instructions
This addresses the challenge of building dialog systems with limited labeled data, though it is incremental as it builds on existing pre-trained models and datasets.
The paper tackles the problem of data scarcity for training dialog systems by using GPT2 to simulate conversations from instructions, achieving significant improvements in low-resource settings on the MultiWOZ and Persona chat datasets.
Popular dialog datasets such as MultiWOZ are created by providing crowd workers an instruction, expressed in natural language, that describes the task to be accomplished. Crowd workers play the role of a user and an agent to generate dialogs to accomplish tasks involving booking restaurant tables, calling a taxi etc. In this paper, we present a data creation strategy that uses the pre-trained language model, GPT2, to simulate the interaction between crowd workers by creating a user bot and an agent bot. We train the simulators using a smaller percentage of actual crowd-generated conversations and their corresponding instructions. We demonstrate that by using the simulated data, we achieve significant improvements in low-resource settings on two publicly available datasets - the MultiWOZ dataset and the Persona chat dataset.