ProST: Progressive Sub-task Training for Pareto-Optimal Multi-agent Systems Using Small Language Models
This work addresses the challenge of deploying efficient multi-agent systems for complex tasks, offering an incremental improvement in training methods for specialized roles.
The paper tackles the problem of multi-agent systems with small language models (SLMs) struggling with long-trajectory learning and subtask errors in complex environments like AppWorld, and finds that a progressive sub-task training strategy improves effectiveness, with Pareto analysis showing better trade-offs in efficiency and effectiveness.
Multi-agent systems with smaller language models (SLMs) present a viable alternative to single agent systems powered by large language models (LLMs) for addressing complex problems. In this work, we study how these alternatives compare in terms of both effectiveness and efficiency. To study this trade-off, we instantiate single and multi-agent systems for the complex problems in the AppWorld environment using different sized language models. We find that difficulties with long-trajectory learning in smaller language models (SLMs) limit their performance. Even when trained for specialized roles, SLMs fail to learn all subtasks effectively. To address this issue, we introduce a simple progressive sub-task training strategy, which introduces new sub-tasks progressively in each training epoch. We find that this novel strategy, analogous to instance level curriculum learning, consistently improves the effectiveness of multi-agents at all configurations. Our Pareto analysis shows that fine-tuned multi-agent systems yield better effectiveness-efficiency trade-offs. Additional ablations and analyses shows the importance of our progressive training strategy and its ability to reduce subtask error rates.