Are LLMs All You Need for Task-Oriented Dialogue?
This work addresses the problem of assessing LLMs' capabilities for multi-turn task-oriented dialogues, providing insights for researchers and practitioners in dialogue systems, though it is incremental as it builds on existing benchmarks and models.
The study evaluated instruction-tuned Large Language Models (LLMs) on task-oriented dialogue benchmarks, finding that they underperform specialized models for explicit belief state tracking but can guide dialogues to successful endings when given correct slot values, with performance improving with access to true belief state distributions or in-domain examples.
Instructions-tuned Large Language Models (LLMs) gained recently huge popularity thanks to their ability to interact with users through conversation. In this work we aim to evaluate their ability to complete multi-turn tasks and interact with external databases in the context of established task-oriented dialogue benchmarks. We show that for explicit belief state tracking, LLMs underperform compared to specialized task-specific models. Nevertheless, they show ability to guide the dialogue to successful ending if given correct slot values. Furthermore this ability improves with access to true belief state distribution or in-domain examples.