ChainCQG: Flow-Aware Conversational Question Generation
This work is significant for researchers and developers of conversational AI systems, as it provides a method to generate synthetic conversations for training and evaluation, addressing a key data bottleneck.
The paper addresses the lack of domain-specific training data for conversational question-answering by focusing on conversational question generation. They propose ChainCQG, a two-stage architecture with a flow propagation training strategy, which achieves up to 48% BLEU-1 improvement over SOTA baselines.
Conversational systems enable numerous valuable applications, and question-answering is an important component underlying many of these. However, conversational question-answering remains challenging due to the lack of realistic, domain-specific training data. Inspired by this bottleneck, we focus on conversational question generation as a means to generate synthetic conversations for training and evaluation purposes. We present a number of novel strategies to improve conversational flow and accommodate varying question types and overall fluidity. Specifically, we design ChainCQG as a two-stage architecture that learns question-answer representations across multiple dialogue turns using a flow propagation training strategy.ChainCQG significantly outperforms both answer-aware and answer-unaware SOTA baselines (e.g., up to 48% BLEU-1 improvement). Additionally, our model is able to generate different types of questions, with improved fluidity and coreference alignment.