Question Generation in Knowledge-Driven Dialog: Explainability and Evaluation
This work addresses explainability and evaluation for question generation in knowledge-driven dialog systems, representing an incremental improvement over existing methods.
The paper tackles question generation in knowledge-grounded dialogs by proposing a model that sequentially predicts a fact and then a question, rather than directly generating a question. It shows that this approach performs on par with a standard model on 37k test dialogs from KGConv, while enabling detailed referenceless evaluation of relevance, factuality, and pronominalisation.
We explore question generation in the context of knowledge-grounded dialogs focusing on explainability and evaluation. Inspired by previous work on planning-based summarisation, we present a model which instead of directly generating a question, sequentially predicts first a fact then a question. We evaluate our approach on 37k test dialogs adapted from the KGConv dataset and we show that, although more demanding in terms of inference, our approach performs on par with a standard model which solely generates a question while allowing for a detailed referenceless evaluation of the model behaviour in terms of relevance, factuality and pronominalisation.