Contextual Out-of-Domain Utterance Handling With Counterfeit Data Augmentation
This addresses the issue of frustrating user experiences in dialog systems due to inappropriate responses to anomalous input, though it is incremental as it builds on prior OOD detection approaches.
The paper tackles the problem of neural dialog models lacking robustness to out-of-domain (OOD) user input by proposing a novel OOD detection method that uses counterfeit OOD turns without requiring OOD data, and it outperforms state-of-the-art models by a large margin in the presence of OOD utterances.
Neural dialog models often lack robustness to anomalous user input and produce inappropriate responses which leads to frustrating user experience. Although there are a set of prior approaches to out-of-domain (OOD) utterance detection, they share a few restrictions: they rely on OOD data or multiple sub-domains, and their OOD detection is context-independent which leads to suboptimal performance in a dialog. The goal of this paper is to propose a novel OOD detection method that does not require OOD data by utilizing counterfeit OOD turns in the context of a dialog. For the sake of fostering further research, we also release new dialog datasets which are 3 publicly available dialog corpora augmented with OOD turns in a controllable way. Our method outperforms state-of-the-art dialog models equipped with a conventional OOD detection mechanism by a large margin in the presence of OOD utterances.