Adversarial and Safely Scaled Question Generation
This addresses the need for scalable and safe question generation for public institutions and applications requiring high accuracy, though it appears incremental as it builds on existing adversarial and pruning techniques.
The paper tackles the dual problems of scaling question generation to massive unlabeled text while ensuring safety against misinformation, hate speech, and profanity, introducing an adversarial approach that prunes unanswerable questions and improves answer quality. It reports generating over six times more quality questions than an abstractive baseline, with a 44% higher perceived quality in a survey of 168 participants.
Question generation has recently gained a lot of research interest, especially with the advent of large language models. In and of itself, question generation can be considered 'AI-hard', as there is a lack of unanimously agreed sense of what makes a question 'good' or 'bad'. In this paper, we tackle two fundamental problems in parallel: on one hand, we try to solve the scaling problem, where question-generation and answering applications have to be applied to a massive amount of text without ground truth labeling. The usual approach to solve this problem is to either downsample or summarize. However, there are critical risks of misinformation with these approaches. On the other hand, and related to the misinformation problem, we try to solve the 'safety' problem, as many public institutions rely on a much higher level of accuracy for the content they provide. We introduce an adversarial approach to tackle the question generation safety problem with scale. Specifically, we designed a question-answering system that specifically prunes out unanswerable questions that may be generated, and further increases the quality of the answers that are generated. We build a production-ready, easily-plugged pipeline that can be used on any given body of text, that is scalable and immune from generating any hate speech, profanity, or misinformation. Based on the results, we are able to generate more than six times the number of quality questions generated by the abstractive approach, with a perceived quality being 44% higher, according to a survey of 168 participants.