Reducing Cognitive Overhead in Tool Use via Multi-Small-Agent Reinforcement Learning
This addresses the issue of unstable coordination in AI systems for tool-use tasks, offering a scalable blueprint for multi-agent design, though it appears incremental as it builds on existing multi-agent and tool-use concepts.
The paper tackled the problem of cognitive-load interference in tool-integrated reasoning systems by introducing MSARL, a framework that decouples reasoning from tool use using multiple small agents, resulting in significant improvements in reasoning stability and final-answer accuracy on mathematical problem solving with code execution.
Recent advances in multi-agent systems highlight the potential of specialized small agents that collaborate via division of labor. Existing tool-integrated reasoning systems, however, often follow a single-agent paradigm in which one large model interleaves long-horizon reasoning with precise tool operations, leading to cognitive-load interference and unstable coordination. We present MSARL, a Multi-Small-Agent Reinforcement Learning framework that explicitly decouples reasoning from tool use. In MSARL, a Reasoning Agent decomposes problems and plans tool invocations, while multiple Tool Agents specialize in specific external tools, each trained via a combination of imitation learning and reinforcement learning with role-specific rewards. On mathematical problem solving with code execution, MSARL significantly improves reasoning stability and final-answer accuracy over single-agent baselines. Moreover, the architecture generalizes to diverse tool-use tasks, demonstrating that cognitive-role decoupling with small agents is a scalable blueprint for multi-agent AI design.