Investigation of Language Understanding Impact for Reinforcement Learning Based Dialogue Systems
This work addresses performance robustness in task-oriented dialogue systems, but it is incremental as it focuses on error analysis within an existing framework.
The study investigated how language understanding errors affect reinforcement learning-based dialogue systems, finding that slot-level errors have more impact on performance than intent-level errors, and the system learned to confirm appropriately for better robustness.
Language understanding is a key component in a spoken dialogue system. In this paper, we investigate how the language understanding module influences the dialogue system performance by conducting a series of systematic experiments on a task-oriented neural dialogue system in a reinforcement learning based setting. The empirical study shows that among different types of language understanding errors, slot-level errors can have more impact on the overall performance of a dialogue system compared to intent-level errors. In addition, our experiments demonstrate that the reinforcement learning based dialogue system is able to learn when and what to confirm in order to achieve better performance and greater robustness.