JMultiWOZ: A Large-Scale Japanese Multi-Domain Task-Oriented Dialogue Dataset
This addresses a gap for researchers and developers of Japanese task-oriented dialogue systems, though it is incremental as it adapts an existing English dataset to a new language.
The authors tackled the lack of a Japanese multi-domain task-oriented dialogue dataset by constructing JMultiWOZ, the first large-scale dataset in Japanese, and demonstrated it provides a benchmark comparable to the English MultiWOZ2.2 while identifying limitations in LLMs for task completion in Japanese.
Dialogue datasets are crucial for deep learning-based task-oriented dialogue system research. While numerous English language multi-domain task-oriented dialogue datasets have been developed and contributed to significant advancements in task-oriented dialogue systems, such a dataset does not exist in Japanese, and research in this area is limited compared to that in English. In this study, towards the advancement of research and development of task-oriented dialogue systems in Japanese, we constructed JMultiWOZ, the first Japanese language large-scale multi-domain task-oriented dialogue dataset. Using JMultiWOZ, we evaluated the dialogue state tracking and response generation capabilities of the state-of-the-art methods on the existing major English benchmark dataset MultiWOZ2.2 and the latest large language model (LLM)-based methods. Our evaluation results demonstrated that JMultiWOZ provides a benchmark that is on par with MultiWOZ2.2. In addition, through evaluation experiments of interactive dialogues with the models and human participants, we identified limitations in the task completion capabilities of LLMs in Japanese.