The AlphaPhysics Term Rewriting System for Marking Algebraic Expressions in Physics Exams
This addresses the challenge of automated assessment in physics education, though it appears incremental as it builds on existing automated reasoning techniques.
The authors tackled the problem of automatically marking typed student answers in Physics exams by developing a system that combines a computer algebra system, an SMT solver, and a term rewriting system, achieving evaluation on over 1500 real-world student responses from the 2023 Australian Physics Olympiad.
We present our method for automatically marking Physics exams. The marking problem consists in assessing typed student answers for correctness with respect to a ground truth solution. This is a challenging problem that we seek to tackle using a combination of a computer algebra system, an SMT solver and a term rewriting system. A Large Language Model is used to interpret and remove errors from student responses and rewrite these in a machine readable format. Once formalized and language-aligned, the next step then consists in applying automated reasoning techniques for assessing student solution correctness. We consider two methods of automated theorem proving: off-the-shelf SMT solving and term rewriting systems tailored for physics problems involving trigonometric expressions. The development of the term rewrite system and establishing termination and confluence properties was not trivial, and we describe it in some detail in the paper. We evaluate our system on a rich pool of over 1500 real-world student exam responses from the 2023 Australian Physics Olympiad.