Distributed Autonomous Swarm Formation for Dynamic Network Bridging
This addresses the challenge of coordinated swarm operations for applications like disaster response, though it appears incremental as it builds on existing MARL and graph methods.
The paper tackles the problem of dynamic network bridging by formulating it as a Decentralized Partially Observable Markov Decision Process (Dec-POMDP) and proposes a Multi-Agent Reinforcement Learning approach based on Graph Convolutional Reinforcement Learning, showing promising results in simulation and near Live Virtual Constructive UAV framework.
Effective operation and seamless cooperation of robotic systems are a fundamental component of next-generation technologies and applications. In contexts such as disaster response, swarm operations require coordinated behavior and mobility control to be handled in a distributed manner, with the quality of the agents' actions heavily relying on the communication between them and the underlying network. In this paper, we formulate the problem of dynamic network bridging in a novel Decentralized Partially Observable Markov Decision Process (Dec-POMDP), where a swarm of agents cooperates to form a link between two distant moving targets. Furthermore, we propose a Multi-Agent Reinforcement Learning (MARL) approach for the problem based on Graph Convolutional Reinforcement Learning (DGN) which naturally applies to the networked, distributed nature of the task. The proposed method is evaluated in a simulated environment and compared to a centralized heuristic baseline showing promising results. Moreover, a further step in the direction of sim-to-real transfer is presented, by additionally evaluating the proposed approach in a near Live Virtual Constructive (LVC) UAV framework.