TOAD: Task-Oriented Automatic Dialogs with Diverse Response Styles
This work addresses the slow and costly data creation problem for virtual assistant developers, though it is incremental as it builds on existing TOD datasets with added stylistic variations.
The authors tackled the challenge of creating high-quality annotated data for Task-Oriented Dialogs (TOD) by introducing TOAD, a novel and scalable dataset with an automatic generation pipeline that simulates realistic app interactions and includes diverse response styles, resulting in benchmarks showing that modeling verbose responses or those without user expression mirroring is more difficult.
In light of recent advances in large language models (LLMs), the expectations for the next generation of virtual assistants include enhanced naturalness and adaptability across diverse usage scenarios. However, the creation of high-quality annotated data for Task-Oriented Dialog (TOD) is recognized to be slow and costly. To address these challenges, we introduce Task-Oriented Automatic Dialogs (TOAD), a novel and scalable TOD dataset along with its automatic generation pipeline. The TOAD dataset simulates realistic app context interaction and provide a variety of system response style options. Two aspects of system response styles are considered, verbosity level and users' expression mirroring. We benchmark TOAD on two response generation tasks, and the results show that modeling more verbose responses or responses without user expression mirroring is more challenging.