A Natural Language Corpus of Common Grounding under Continuous and Partially-Observable Context
This addresses the need for better evaluation and analysis of dialogue systems' common grounding abilities, though it is incremental as it introduces a new task and dataset rather than a breakthrough method.
The authors tackled the problem of limited common grounding in dialogue systems by proposing a minimal dialogue task requiring advanced common grounding under continuous and partially-observable contexts, resulting in a dataset of 6,760 dialogues and showing that baseline neural models perform decently but have room for improvement.
Common grounding is the process of creating, repairing and updating mutual understandings, which is a critical aspect of sophisticated human communication. However, traditional dialogue systems have limited capability of establishing common ground, and we also lack task formulations which introduce natural difficulty in terms of common grounding while enabling easy evaluation and analysis of complex models. In this paper, we propose a minimal dialogue task which requires advanced skills of common grounding under continuous and partially-observable context. Based on this task formulation, we collected a largescale dataset of 6,760 dialogues which fulfills essential requirements of natural language corpora. Our analysis of the dataset revealed important phenomena related to common grounding that need to be considered. Finally, we evaluate and analyze baseline neural models on a simple subtask that requires recognition of the created common ground. We show that simple baseline models perform decently but leave room for further improvement. Overall, we show that our proposed task will be a fundamental testbed where we can train, evaluate, and analyze dialogue system's ability for sophisticated common grounding.