Model Capability Dominates: Inference-Time Optimization Lessons from AIMO 3
For practitioners using LLMs for mathematical reasoning, this paper shows that investing in stronger base models is far more effective than complex inference-time strategies.
The paper tests diverse prompting strategies to decorrelate errors in majority voting for mathematical reasoning, but finds that high-temperature sampling already achieves sufficient decorrelation, and weaker prompts reduce per-attempt accuracy more than they reduce correlation. Across a 17-point model capability gap, model capability dominates inference-time optimizations by an order of magnitude.
Majority voting over multiple LLM attempts improves mathematical reasoning, but correlated errors limit the effective sample size. A natural fix: assign structurally different reasoning strategies to different voters to decorrelate errors. We test this Diverse Prompt Mixer in the AIMO~3 competition: 3 models, 23+ experiments, and 50 IMO-level problems on a single H100 80 GB with a 5-hour limit. Every intervention fails. High-temperature sampling already decorrelates errors sufficiently; weaker prompt strategies reduce per-attempt accuracy more than they reduce correlation. Across a 17-point model capability gap and every inference-time optimization we tried, model capability dominates by an order of magnitude.