A Large Collection of Model-generated Contradictory Responses for Consistency-aware Dialogue Systems
This work addresses a critical issue in dialogue systems for improving response consistency, though it is incremental as it focuses on dataset creation rather than a novel method.
The paper tackles the problem of contradictory responses in dialogue systems by building the first large dataset of model-generated contradictory responses, which enables comprehensive analysis of their characteristics and improves the performance of data-driven contradiction suppression methods by a substantial margin.
Mitigating the generation of contradictory responses poses a substantial challenge in dialogue response generation. The quality and quantity of available contradictory response data play a vital role in suppressing these contradictions, offering two significant benefits. First, having access to large contradiction data enables a comprehensive examination of their characteristics. Second, data-driven methods to mitigate contradictions may be enhanced with large-scale contradiction data for training. Nevertheless, no attempt has been made to build an extensive collection of model-generated contradictory responses. In this paper, we build a large dataset of response generation models' contradictions for the first time. Then, we acquire valuable insights into the characteristics of model-generated contradictions through an extensive analysis of the collected responses. Lastly, we also demonstrate how this dataset substantially enhances the performance of data-driven contradiction suppression methods.