EasyMath: A 0-shot Math Benchmark for SLMs
This work addresses the need for a practical math reasoning benchmark for small language models, though it is incremental as it builds on existing benchmarking approaches.
The authors introduced EasyMath, a compact benchmark for evaluating practical math reasoning in small language models across thirteen categories, and tested 23 models (14M to 4B parameters) in a zero-shot setting, finding that accuracy improves with model size and training, with chain-of-thought providing modest gains.
EasyMath is a compact benchmark for practical math reasoning in small language models. It covers thirteen categories, from basic arithmetic and order of operations to word problems, algebraic expressions, edge cases, and omits specialist topics. We tested 23 models (14M to 4B parameters) using exact, numerical, and symbolic checks on free-form answers in a zero-shot setting. Accuracy rises with size and training, chain-of-thought adds modest gains, and consistency improves at scale.