Steps are all you need: Rethinking STEM Education with Prompt Engineering
This addresses physics education challenges by enhancing LLM accuracy, though it appears incremental in prompt engineering methods.
The paper tackles the problem of limited mathematical ability and hallucination in LLMs for physics question answering by using a Mixture of Experts model with analogical prompting, showing improved performance compared to baseline LLMs. It also proposes Analogical CoT prompting to help smaller open-source models leverage analogical techniques.
Few shot and Chain-of-Thought prompting have shown promise when applied to Physics Question Answering Tasks, but are limited by the lack of mathematical ability inherent to LLMs, and are prone to hallucination. By utilizing a Mixture of Experts (MoE) Model, along with analogical prompting, we are able to show improved model performance when compared to the baseline on standard LLMs. We also survey the limits of these prompting techniques and the effects they have on model performance. Additionally, we propose Analogical CoT prompting, a prompting technique designed to allow smaller, open source models to leverage Analogical prompting, something they have struggled with, possibly due to a lack of specialist training data.