Physics Reasoner: Knowledge-Augmented Reasoning for Solving Physics Problems with Large Language Models
This addresses the challenge of improving LLMs' reasoning in physics for educational or AI applications, but it is incremental as it builds on existing methods with specific enhancements.
The paper tackles the problem of large language models (LLMs) failing at physics problems due to insufficient knowledge or incorrect application, proposing Physics Reasoner, a knowledge-augmented framework that achieves state-of-the-art performance on SciBench with a 5.8% average accuracy improvement.
Physics problems constitute a significant aspect of reasoning, necessitating complicated reasoning ability and abundant physics knowledge. However, existing large language models (LLMs) frequently fail due to a lack of knowledge or incorrect knowledge application. To mitigate these issues, we propose Physics Reasoner, a knowledge-augmented framework to solve physics problems with LLMs. Specifically, the proposed framework constructs a comprehensive formula set to provide explicit physics knowledge and utilizes checklists containing detailed instructions to guide effective knowledge application. Namely, given a physics problem, Physics Reasoner solves it through three stages: problem analysis, formula retrieval, and guided reasoning. During the process, checklists are employed to enhance LLMs' self-improvement in the analysis and reasoning stages. Empirically, Physics Reasoner mitigates the issues of insufficient knowledge and incorrect application, achieving state-of-the-art performance on SciBench with an average accuracy improvement of 5.8%.