An Analysis of Mixed Initiative and Collaboration in Information-Seeking Dialogues
This work addresses the challenge of improving conversational search systems for users by analyzing dialogue interactions, though it appears incremental as it builds on existing datasets and metrics without introducing a new paradigm.
The authors tackled the problem of understanding mixed-initiative interaction in conversational search systems by proposing unsupervised metrics called ConversationShape to analyze dialogue roles and vocabulary distributions. They found that deviations from human-human dialogue patterns predict the quality of human-machine dialogues, with specific predictive results implied but not quantified in the abstract.
The ability to engage in mixed-initiative interaction is one of the core requirements for a conversational search system. How to achieve this is poorly understood. We propose a set of unsupervised metrics, termed ConversationShape, that highlights the role each of the conversation participants plays by comparing the distribution of vocabulary and utterance types. Using ConversationShape as a lens, we take a closer look at several conversational search datasets and compare them with other dialogue datasets to better understand the types of dialogue interaction they represent, either driven by the information seeker or the assistant. We discover that deviations from the ConversationShape of a human-human dialogue of the same type is predictive of the quality of a human-machine dialogue.