Measuring Semantic Coherence of a Conversation
This work addresses the need for conversational systems to model semantics accurately, but it is incremental as it builds on existing methods for coherence measurement.
The paper tackles the problem of measuring semantic coherence in conversations by introducing a task that identifies semantic relations between concepts using background knowledge, and proposes graph-based and machine learning approaches evaluated on the Ubuntu Dialogue Corpus to uncover different coherence patterns.
Conversational systems have become increasingly popular as a way for humans to interact with computers. To be able to provide intelligent responses, conversational systems must correctly model the structure and semantics of a conversation. We introduce the task of measuring semantic (in)coherence in a conversation with respect to background knowledge, which relies on the identification of semantic relations between concepts introduced during a conversation. We propose and evaluate graph-based and machine learning-based approaches for measuring semantic coherence using knowledge graphs, their vector space embeddings and word embedding models, as sources of background knowledge. We demonstrate how these approaches are able to uncover different coherence patterns in conversations on the Ubuntu Dialogue Corpus.