Challenge Closed-book Science Exam: A Meta-learning Based Question Answering System
This addresses the challenge of time-consuming knowledge retrieval in question answering for standardized science exams, offering a competitive alternative to retrieval-based methods.
The paper tackles the problem of answering closed-book science exam questions without relying on external knowledge bases, proposing a MetaQA framework that improves reasoning module accuracy from 46.6% to 64.2% on the AI2 Reasoning Challenge.
Prior work in standardized science exams requires support from large text corpus, such as targeted science corpus fromWikipedia or SimpleWikipedia. However, retrieving knowledge from the large corpus is time-consuming and questions embedded in complex semantic representation may interfere with retrieval. Inspired by the dual process theory in cognitive science, we propose a MetaQA framework, where system 1 is an intuitive meta-classifier and system 2 is a reasoning module. Specifically, our method based on meta-learning method and large language model BERT, which can efficiently solve science problems by learning from related example questions without relying on external knowledge bases. We evaluate our method on AI2 Reasoning Challenge (ARC), and the experimental results show that meta-classifier yields considerable classification performance on emerging question types. The information provided by meta-classifier significantly improves the accuracy of reasoning module from 46.6% to 64.2%, which has a competitive advantage over retrieval-based QA methods.