Analyzing and Characterizing User Intent in Information-seeking Conversations
This work addresses the need for better conversational search systems by providing a dataset and analysis for researchers, though it is incremental as it focuses on data creation and characterization rather than novel methods.
The paper tackles the problem of understanding user intent in information-seeking conversations by introducing the MSDialog dataset, a labeled collection of over 2,000 multi-turn QA dialogs with 10,000 utterances annotated for user intent, and uses it to analyze recurring patterns in intent distribution, co-occurrence, and flow.
Understanding and characterizing how people interact in information-seeking conversations is crucial in developing conversational search systems. In this paper, we introduce a new dataset designed for this purpose and use it to analyze information-seeking conversations by user intent distribution, co-occurrence, and flow patterns. The MSDialog dataset is a labeled dialog dataset of question answering (QA) interactions between information seekers and providers from an online forum on Microsoft products. The dataset contains more than 2,000 multi-turn QA dialogs with 10,000 utterances that are annotated with user intent on the utterance level. Annotations were done using crowdsourcing. With MSDialog, we find some highly recurring patterns in user intent during an information-seeking process. They could be useful for designing conversational search systems. We will make our dataset freely available to encourage exploration of information-seeking conversation models.