Automated Feedback Generation for a Chemistry Database and Abstracting Exercise
This work addresses a specific teaching challenge in chemistry education by automating feedback, but it is incremental as it applies an existing method to a new dataset.
The authors tackled the problem of providing timely feedback on student summaries of chemistry papers by using a BERT model to classify sentences into background, technique, and observation categories, revealing that students focus more on background and less on techniques and results compared to PubMed abstracts, and enabling automated feedback generation.
Timely feedback is an important part of teaching and learning. Here we describe how a readily available neural network transformer (machine-learning) model (BERT) can be used to give feedback on the structure of the response to an abstracting exercise where students are asked to summarise the contents of a published article after finding it from a publication database. The dataset contained 207 submissions from two consecutive years of the course, summarising a total of 21 different papers from the primary literature. The model was pre-trained using an available dataset (approx. 15,000 samples) and then fine-tuned on 80% of the submitted dataset. This fine tuning was seen to be important. The sentences in the student submissions are characterised into three classes - background, technique and observation - which allows a comparison of how each submission is structured. Comparing the structure of the students' abstract a large collection of those from the PubMed database shows that students in this exercise concentrate more on the background to the paper and less on the techniques and results than the abstracts to papers themselves. The results allowed feedback for each submitted assignment to be automatically generated.