HEART: Coordination of Heterogeneous Expert Agents for Physically Grounded Robotic Task Planning
For robotic task planning, HEART addresses the problem of LLMs failing to satisfy physical and spatial constraints, offering a scalable solution under resource constraints.
HEART introduces a heterogeneous multi-LLM framework for robotic task planning that decomposes instructions into atomic reasoning tasks assigned to role-specialized agents, achieving improved plan success over single-LLM and rule-based planners in household benchmarks.
Large Language Models (LLMs) can reason over complex instructions but often fail to satisfy the physical and spatial constraints required for robotic task planning. Recent LLM-based planners directly translate text into action sequences, yet they lack structured reasoning about feasibility, reachability, and logical order, resulting in invalid or incomplete plans. We present a heterogeneous multi-LLM framework that decomposes instructions into atomic reasoning tasks and allocates them to role-specialized expert agents under a token budget for real-world computational and communicational constraints. By combining role-oriented reasoning from heterogeneous agents followed by constraint-driven plan synthesis, HEART validates capability, reachability, and constraint conditions before planning and helps produce physically executable plans while maintaining efficiency. Experiments across different household benchmarks show that HEART consistently improves plan success compared to single-LLM and rule-based planners, demonstrating that heterogeneous LLM collaboration enables robust and scalable robotic task planning under resource constraints.