Simplifying Paragraph-level Question Generation via Transformer Language Models
This work addresses the need for simpler and more practical question generation models for NLP applications, though it is incremental as it builds on existing Transformer methods.
The authors tackled the problem of generating questions from paragraphs by simplifying existing complex models, showing that a GPT-2 Small finetuned model outperforms baselines on SQuAD by 0.95 METEOR points and produces human-rated high-quality questions.
Question generation (QG) is a natural language generation task where a model is trained to ask questions corresponding to some input text. Most recent approaches frame QG as a sequence-to-sequence problem and rely on additional features and mechanisms to increase performance; however, these often increase model complexity, and can rely on auxiliary data unavailable in practical use. A single Transformer-based unidirectional language model leveraging transfer learning can be used to produce high quality questions while disposing of additional task-specific complexity. Our QG model, finetuned from GPT-2 Small, outperforms several paragraph-level QG baselines on the SQuAD dataset by 0.95 METEOR points. Human evaluators rated questions as easy to answer, relevant to their context paragraph, and corresponding well to natural human speech. Also introduced is a new set of baseline scores on the RACE dataset, which has not previously been used for QG tasks. Further experimentation with varying model capacities and datasets with non-identification type questions is recommended in order to further verify the robustness of pretrained Transformer-based LMs as question generators.