RoMath: A Mathematical Reasoning Benchmark in Romanian
It addresses the problem of limited mathematical reasoning benchmarks for low-resource languages like Romanian, which is incremental as it extends existing benchmark efforts to a new language.
The paper introduces RoMath, a Romanian mathematical reasoning benchmark suite covering three subsets to address the lack of non-English resources, and benchmarks open-weight language models to highlight the need for dedicated resources beyond translation.
Mathematics has long been conveyed through natural language, primarily for human understanding. With the rise of mechanized mathematics and proof assistants, there is a growing need to understand informal mathematical text, yet most existing benchmarks focus solely on English, overlooking other languages. This paper introduces RoMath, a Romanian mathematical reasoning benchmark suite comprising three subsets: Baccalaureate, Competitions and Synthetic, which cover a range of mathematical domains and difficulty levels, aiming to improve non-English language models and promote multilingual AI development. By focusing on Romanian, a low-resource language with unique linguistic features, RoMath addresses the limitations of Anglo-centric models and emphasizes the need for dedicated resources beyond simple automatic translation. We benchmark several open-weight language models, highlighting the importance of creating resources for underrepresented languages. Code and datasets are be made available.