FOLLOWUPQG: Towards Information-Seeking Follow-up Question Generation
This addresses the need for more realistic follow-up question generation in conversational AI, though it is incremental as it builds on existing question generation tasks.
The paper tackles the problem of generating follow-up questions for information-seeking dialogues by introducing the FOLLOWUPQG dataset with over 3K real-world examples, and finds that current models produce adequate but less informative and complex questions compared to humans.
Humans ask follow-up questions driven by curiosity, which reflects a creative human cognitive process. We introduce the task of real-world information-seeking follow-up question generation (FQG), which aims to generate follow-up questions seeking a more in-depth understanding of an initial question and answer. We construct FOLLOWUPQG, a dataset of over 3K real-world (initial question, answer, follow-up question) tuples collected from a Reddit forum providing layman-friendly explanations for open-ended questions. In contrast to existing datasets, questions in FOLLOWUPQG use more diverse pragmatic strategies to seek information, and they also show higher-order cognitive skills (such as applying and relating). We evaluate current question generation models on their efficacy for generating follow-up questions, exploring how to generate specific types of follow-up questions based on step-by-step demonstrations. Our results validate FOLLOWUPQG as a challenging benchmark, as model-generated questions are adequate but far from human-raised questions in terms of informativeness and complexity.