We've had this conversation before: A Novel Approach to Measuring Dialog Similarity
This work addresses the need for improved dialog comparison in applications like conversation analytics and virtual agent design, representing an incremental advancement over existing similarity measures.
The paper tackles the problem of measuring dialog similarity by proposing a novel adaptation of edit distance that accounts for utterance semantics, conversation flow, and participants, and demonstrates that it outperforms existing document similarity methods on two datasets by better capturing dialog flow and aligning with human perception.
Dialog is a core building block of human natural language interactions. It contains multi-party utterances used to convey information from one party to another in a dynamic and evolving manner. The ability to compare dialogs is beneficial in many real world use cases, such as conversation analytics for contact center calls and virtual agent design. We propose a novel adaptation of the edit distance metric to the scenario of dialog similarity. Our approach takes into account various conversation aspects such as utterance semantics, conversation flow, and the participants. We evaluate this new approach and compare it to existing document similarity measures on two publicly available datasets. The results demonstrate that our method outperforms the other approaches in capturing dialog flow, and is better aligned with the human perception of conversation similarity.