Heterogeneous Multi-Agent Reinforcement Learning for Unknown Environment Mapping
This work addresses the need for enhanced situational awareness in hazardous areas, such as for national security and emergency response, by deploying teams of unmanned aerial vehicles, though it is incremental in its approach.
The paper tackles the problem of enabling heterogeneous multi-agent teams to learn decentralized control policies for covering unknown environments, achieving improved coverage efficiency and robustness to real-world disturbances like turbulence and communication delays.
Reinforcement learning in heterogeneous multi-agent scenarios is important for real-world applications but presents challenges beyond those seen in homogeneous settings and simple benchmarks. In this work, we present an actor-critic algorithm that allows a team of heterogeneous agents to learn decentralized control policies for covering an unknown environment. This task is of interest to national security and emergency response organizations that would like to enhance situational awareness in hazardous areas by deploying teams of unmanned aerial vehicles. To solve this multi-agent coverage path planning problem in unknown environments, we augment a multi-agent actor-critic architecture with a new state encoding structure and triplet learning loss to support heterogeneous agent learning. We developed a simulation environment that includes real-world environmental factors such as turbulence, delayed communication, and agent loss, to train teams of agents as well as probe their robustness and flexibility to such disturbances.