Communication-Aware Multi-Agent Reinforcement Learning for Decentralized Cooperative UAV Deployment
This addresses the challenge of scalable and robust UAV swarm operations for applications like aerial relays and sensing, though it is incremental as it builds on existing CTDE and attention mechanisms.
The paper tackles the problem of decentralized cooperative UAV deployment under partial observability and intermittent communication by proposing a graph-based multi-agent reinforcement learning framework, achieving 74% coverage with 5 UAVs and 10 nodes in a relay task and improved win rates in an adversarial setting.
Autonomous Unmanned Aerial Vehicle (UAV) swarms are increasingly used as rapidly deployable aerial relays and sensing platforms, yet practical deployments must operate under partial observability and intermittent peer-to-peer links. We present a graph-based multi-agent reinforcement learning framework trained under centralized training with decentralized execution (CTDE): a centralized critic and global state are available only during training, while each UAV executes a shared policy using local observations and messages from nearby neighbors. Our architecture encodes local agent state and nearby entities with an agent-entity attention module, and aggregates inter-UAV messages with neighbor self-attention over a distance-limited communication graph. We evaluate primarily on a cooperative relay deployment task (DroneConnect) and secondarily on an adversarial engagement task (DroneCombat). In DroneConnect, the proposed method achieves high coverage under restricted communication and partial observation (e.g. 74% coverage with M = 5 UAVs and N = 10 nodes) while remaining competitive with a mixed-integer linear programming (MILP) optimization-based offline upper bound, and it generalizes to unseen team sizes without fine-tuning. In the adversarial setting, the same framework transfers without architectural changes and improves win rate over non-communicating baselines.