Learning Beyond Pattern Matching? Assaying Mathematical Understanding in LLMs
It addresses the problem of evaluating LLMs' true comprehension for scientific applications, but the findings are incremental as they build on existing methods like NTK.
This paper assesses the mathematical understanding of large language models (LLMs) by analyzing how they learn from different types of math data during in-context learning and instruction-tuning, finding evidence of domain understanding in in-context learning but a lack of it in certain instruction-tuning.
We are beginning to see progress in language model assisted scientific discovery. Motivated by the use of LLMs as a general scientific assistant, this paper assesses the domain knowledge of LLMs through its understanding of different mathematical skills required to solve problems. In particular, we look at not just what the pre-trained model already knows, but how it learned to learn from information during in-context learning or instruction-tuning through exploiting the complex knowledge structure within mathematics. Motivated by the Neural Tangent Kernel (NTK), we propose \textit{NTKEval} to assess changes in LLM's probability distribution via training on different kinds of math data. Our systematic analysis finds evidence of domain understanding during in-context learning. By contrast, certain instruction-tuning leads to similar performance changes irrespective of training on different data, suggesting a lack of domain understanding across different skills.