Math Takes Two: A test for emergent mathematical reasoning in communication
For AI researchers evaluating whether language models exhibit genuine mathematical reasoning versus pattern matching, this benchmark provides a more fundamental test by requiring communication-based discovery of numerical concepts.
The paper introduces Math Takes Two, a benchmark that tests whether two agents can develop a shared symbolic protocol for numerical reasoning from scratch, without prior mathematical knowledge. Results show that current models struggle to discover latent numerical structure, highlighting a gap in emergent reasoning capabilities.
Although language models demonstrate remarkable proficiency on mathematical benchmarks, it remains unclear whether this reflects true mathematical reasoning or statistical pattern matching over learning formal syntax. Most existing evaluations rely on symbolic problems grounded in established mathematical conventions, limiting insight into the models' ability to construct abstract concepts from first principles. In this work, we propose Math Takes Two, a new benchmark designed to assess the emergence of mathematical reasoning through communication. Motivated by the hypothesis that mathematical cognition in humans co-evolved with the need for precise communication, our benchmark tests whether two agents, without prior mathematical knowledge, can develop a shared symbolic protocol to solve a visually grounded task where the use of a numerical system facilitates extrapolation. Unlike many current datasets, our benchmark eschews predefined mathematical language, instead requiring agents to discover latent structure and representations from scratch. Math Takes Two thus provides a novel lens through which to develop and evaluate models with emergent numerical reasoning capabilities.