Learning to Ask Good Questions: Ranking Clarification Questions using Neural Expected Value of Perfect Information
This work addresses the challenge of enabling machines to ask effective questions for better human-machine collaboration, though it is incremental as it builds on existing concepts with a new dataset and model.
The paper tackled the problem of ranking clarification questions by developing a neural network model based on the expected value of perfect information, using data from StackExchange. It demonstrated significant improvements over baselines when evaluated against expert human judgments on a dataset of ~77K posts.
Inquiry is fundamental to communication, and machines cannot effectively collaborate with humans unless they can ask questions. In this work, we build a neural network model for the task of ranking clarification questions. Our model is inspired by the idea of expected value of perfect information: a good question is one whose expected answer will be useful. We study this problem using data from StackExchange, a plentiful online resource in which people routinely ask clarifying questions to posts so that they can better offer assistance to the original poster. We create a dataset of clarification questions consisting of ~77K posts paired with a clarification question (and answer) from three domains of StackExchange: askubuntu, unix and superuser. We evaluate our model on 500 samples of this dataset against expert human judgments and demonstrate significant improvements over controlled baselines.