Hierarchical Reinforcement Learning with Low-Level MPC for Multi-Agent Control
This addresses the problem of reliable multi-agent control for robotics or autonomous systems, representing an incremental improvement by combining existing methods.
The paper tackles the challenge of achieving safe and coordinated behavior in dynamic, constraint-rich multi-agent environments by proposing a hierarchical framework that combines reinforcement learning for tactical decision-making with Model Predictive Control for low-level execution. The approach outperforms end-to-end and shielding-based RL baselines on a predator-prey benchmark in terms of reward, safety, and consistency.
Achieving safe and coordinated behavior in dynamic, constraint-rich environments remains a major challenge for learning-based control. Pure end-to-end learning often suffers from poor sample efficiency and limited reliability, while model-based methods depend on predefined references and struggle to generalize. We propose a hierarchical framework that combines tactical decision-making via reinforcement learning (RL) with low-level execution through Model Predictive Control (MPC). For the case of multi-agent systems this means that high-level policies select abstract targets from structured regions of interest (ROIs), while MPC ensures dynamically feasible and safe motion. Tested on a predator-prey benchmark, our approach outperforms end-to-end and shielding-based RL baselines in terms of reward, safety, and consistency, underscoring the benefits of combining structured learning with model-based control.