SkyMath: Technical Report
This work addresses the challenge of improving mathematical reasoning in NLP for researchers and practitioners, but it appears incremental as it builds on existing base models with fine-tuning.
The authors tackled the problem of enhancing mathematical reasoning in large language models by developing SkyMath, a 13-billion-parameter model that outperforms all known open-source models of similar size on GSM8K, establishing a new state-of-the-art performance.
Large language models (LLMs) have shown great potential to solve varieties of natural language processing (NLP) tasks, including mathematical reasoning. In this work, we present SkyMath, a large language model for mathematics with 13 billion parameters. By applying self-compare fine-tuning, we have enhanced mathematical reasoning abilities of Skywork-13B-Base remarkably. On GSM8K, SkyMath outperforms all known open-source models of similar size and has established a new SOTA performance.