Sources of Noise in Dialogue and How to Deal with Them
This addresses the issue of noisy training data for dialogue system developers, but it is incremental as it builds on existing denoising methods.
The paper tackled the problem of noise in dialogue systems by constructing a taxonomy of noise types and experimentally showing that models are robust to label errors but suffer from dialogue-specific noise, leading to the design of a specialized data cleaning algorithm.
Training dialogue systems often entails dealing with noisy training examples and unexpected user inputs. Despite their prevalence, there currently lacks an accurate survey of dialogue noise, nor is there a clear sense of the impact of each noise type on task performance. This paper addresses this gap by first constructing a taxonomy of noise encountered by dialogue systems. In addition, we run a series of experiments to show how different models behave when subjected to varying levels of noise and types of noise. Our results reveal that models are quite robust to label errors commonly tackled by existing denoising algorithms, but that performance suffers from dialogue-specific noise. Driven by these observations, we design a data cleaning algorithm specialized for conversational settings and apply it as a proof-of-concept for targeted dialogue denoising.