Conversation Graph: Data Augmentation, Training and Evaluation for Non-Deterministic Dialogue Management
This work addresses data scarcity and training inefficiencies for non-deterministic task-oriented dialogue systems, offering a practical solution with measurable gains, though it is incremental as it builds on existing graph-based and data augmentation methods.
The paper tackles the problem of limited and low-diversity training data for non-deterministic dialogue systems, where multiple actions can be valid in the same state, by proposing the Conversation Graph (ConvGraph) for data augmentation and multi-reference training, resulting in improvements in dialogue success rates by up to 6.4% across three datasets.
Task-oriented dialogue systems typically rely on large amounts of high-quality training data or require complex handcrafted rules. However, existing datasets are often limited in size considering the complexity of the dialogues. Additionally, conventional training signal inference is not suitable for non-deterministic agent behaviour, i.e. considering multiple actions as valid in identical dialogue states. We propose the Conversation Graph (ConvGraph), a graph-based representation of dialogues that can be exploited for data augmentation, multi-reference training and evaluation of non-deterministic agents. ConvGraph generates novel dialogue paths to augment data volume and diversity. Intrinsic and extrinsic evaluation across three datasets shows that data augmentation and/or multi-reference training with ConvGraph can improve dialogue success rates by up to 6.4%.