PRESTO: A Multilingual Dataset for Parsing Realistic Task-Oriented Dialogs
This addresses the lack of realistic datasets for parsing task-oriented dialogs, enabling research on challenging NLU aspects, though it is incremental as it focuses on data creation.
The authors introduced PRESTO, a dataset of over 550K multilingual human-assistant conversations designed to capture real-world challenges like disfluencies and code-switching, and found that mT5 baselines struggle with these phenomena, especially in low-resource settings.
Research interest in task-oriented dialogs has increased as systems such as Google Assistant, Alexa and Siri have become ubiquitous in everyday life. However, the impact of academic research in this area has been limited by the lack of datasets that realistically capture the wide array of user pain points. To enable research on some of the more challenging aspects of parsing realistic conversations, we introduce PRESTO, a public dataset of over 550K contextual multilingual conversations between humans and virtual assistants. PRESTO contains a diverse array of challenges that occur in real-world NLU tasks such as disfluencies, code-switching, and revisions. It is the only large scale human generated conversational parsing dataset that provides structured context such as a user's contacts and lists for each example. Our mT5 model based baselines demonstrate that the conversational phenomenon present in PRESTO are challenging to model, which is further pronounced in a low-resource setup.