Contextual Semantic Parsing for Multilingual Task-Oriented Dialogues
This enables high-accuracy multilingual dialogue systems without expensive human annotations, though it's an incremental improvement over existing translation-based methods.
The paper tackles the problem of limited multilingual support in task-oriented dialogue systems by automatically translating dialogue datasets with alignment to preserve slot values, and proposes a contextual semantic parsing model that reduces translation error propagation. The approach improves state-of-the-art joint goal accuracy by up to 20% on several benchmarks and creates new English and German datasets with accuracy within 11% of the original.
Robust state tracking for task-oriented dialogue systems currently remains restricted to a few popular languages. This paper shows that given a large-scale dialogue data set in one language, we can automatically produce an effective semantic parser for other languages using machine translation. We propose automatic translation of dialogue datasets with alignment to ensure faithful translation of slot values and eliminate costly human supervision used in previous benchmarks. We also propose a new contextual semantic parsing model, which encodes the formal slots and values, and only the last agent and user utterances. We show that the succinct representation reduces the compounding effect of translation errors, without harming the accuracy in practice. We evaluate our approach on several dialogue state tracking benchmarks. On RiSAWOZ, CrossWOZ, CrossWOZ-EN, and MultiWOZ-ZH datasets we improve the state of the art by 11%, 17%, 20%, and 0.3% in joint goal accuracy. We present a comprehensive error analysis for all three datasets showing erroneous annotations can lead to misguided judgments on the quality of the model. Finally, we present RiSAWOZ English and German datasets, created using our translation methodology. On these datasets, accuracy is within 11% of the original showing that high-accuracy multilingual dialogue datasets are possible without relying on expensive human annotations. We release our datasets and software open source.