Estimating the Usefulness of Clarifying Questions and Answers for Conversational Search
This work addresses a specific bottleneck in conversational search systems for users, but it is incremental as it builds on existing query rewriting methods.
The paper tackled the problem of processing users' answers to clarifying questions in conversational search, which is understudied compared to question generation, by proposing a classifier to assess answer usefulness and integrating useful ones into query rewriting, resulting in significant improvements over strong baselines and mitigating performance drops from non-useful content.
While the body of research directed towards constructing and generating clarifying questions in mixed-initiative conversational search systems is vast, research aimed at processing and comprehending users' answers to such questions is scarce. To this end, we present a simple yet effective method for processing answers to clarifying questions, moving away from previous work that simply appends answers to the original query and thus potentially degrades retrieval performance. Specifically, we propose a classifier for assessing usefulness of the prompted clarifying question and an answer given by the user. Useful questions or answers are further appended to the conversation history and passed to a transformer-based query rewriting module. Results demonstrate significant improvements over strong non-mixed-initiative baselines. Furthermore, the proposed approach mitigates the performance drops when non useful questions and answers are utilized.