Analyzing Information Sharing and Coordination in Multi-Agent Planning
This addresses the problem of error reduction in multi-agent planning for LLM-based systems, though it is incremental as it builds on existing MAS frameworks.
The study tackled the challenge of long-horizon, multi-constraint planning in multi-agent systems by evaluating a notebook for information sharing and an orchestrator for coordination in a travel planning task, resulting in a 25% final pass rate, a 17.5% absolute improvement over the baseline.
Multi-agent systems (MASs) have pushed the boundaries of large language model (LLM) agents in domains such as web research and software engineering. However, long-horizon, multi-constraint planning tasks involve conditioning on detailed information and satisfying complex interdependent constraints, which can pose a challenge for these systems. In this study, we construct an LLM-based MAS for a travel planning task which is representative of these challenges. We evaluate the impact of a notebook to facilitate information sharing, and evaluate an orchestrator agent to improve coordination in free form conversation between agents. We find that the notebook reduces errors due to hallucinated details by 18%, while an orchestrator directs the MAS to focus on and further reduce errors by up to 13.5% within focused sub-areas. Combining both mechanisms achieves a 25% final pass rate on the TravelPlanner benchmark, a 17.5% absolute improvement over the single-agent baseline's 7.5% pass rate. These results highlight the potential of structured information sharing and reflective orchestration as key components in MASs for long horizon planning with LLMs.