Learning from Relevant Subgoals in Successful Dialogs using Iterative Training for Task-oriented Dialog Systems
This addresses the challenge of improving dialog success rates for users in task-oriented systems, though it is incremental as it builds on existing training methods.
The paper tackles the problem of task-oriented dialog systems lacking intermediate feedback by proposing SUIT, an iterative training approach that samples dialogs and identifies relevant subgoals to generate high-quality training data, achieving new state-of-the-art performance on a popular benchmark.
Task-oriented Dialog (ToD) systems have to solve multiple subgoals to accomplish user goals, whereas feedback is often obtained only at the end of the dialog. In this work, we propose SUIT (SUbgoal-aware ITerative Training), an iterative training approach for improving ToD systems. We sample dialogs from the model we aim to improve and determine subgoals that contribute to dialog success using distant supervision to obtain high quality training samples. We show how this data improves supervised fine-tuning or, alternatively, preference learning results. SUIT is able to iteratively generate more data instead of relying on fixed static sets. SUIT reaches new state-of-the-art performance on a popular ToD benchmark.