Constructing Open Cloze Tests Using Generation and Discrimination Capabilities of Transformers
This work addresses the need for automated test generation in education or NLP applications, but it appears incremental as it builds on existing transformer models with specific enhancements.
The paper tackles the problem of constructing open cloze tests by developing a multi-objective transformer model that combines generation and discrimination capabilities, achieving up to 82% accuracy in expert evaluations and outperforming prior methods.
This paper presents the first multi-objective transformer model for constructing open cloze tests that exploits generation and discrimination capabilities to improve performance. Our model is further enhanced by tweaking its loss function and applying a post-processing re-ranking algorithm that improves overall test structure. Experiments using automatic and human evaluation show that our approach can achieve up to 82% accuracy according to experts, outperforming previous work and baselines. We also release a collection of high-quality open cloze tests along with sample system output and human annotations that can serve as a future benchmark.