AGORA: Incentivizing Group Emergence Capability in LLMs via Group Distillation
This addresses the problem of constrained reasoning progress in LLMs by demonstrating scalable group emergence, though it appears incremental as an extension of ensemble methods.
The paper tackles the limitation of static training datasets in complex reasoning by proposing structured interaction as a new scaling axis, achieving up to 4.45 percentage point improvements over state-of-the-art monolithic systems on challenging mathematical benchmarks.
Progress in complex reasoning is constrained by the static nature of the current training datasets. We propose structured interaction as a new scaling axis, moving beyond the prevailing paradigm of increasing model parameters. Our self-evolving framework, AGORA, enables a collaborative ensemble to achieve reasoning performance exceeding state-of-the-art monolithic systems by up to 4.45 percentage points on challenging mathematical benchmarks. This gain stems from group emergent ability-the synthesis of collective capabilities unattainable by isolated models, validating interaction as a scalable driver of intelligence. Our results position the engineering of collaborative ecosystems as a vital frontier for capability emergence.