Foresight of Graph Reinforcement Learning Latent Permutations Learnt by Gumbel Sinkhorn Network
This work addresses the problem of improving cooperation in complex, dynamic multi-agent systems for AI and robotics applications, but it appears incremental as it builds on existing graph neural network and reinforcement learning techniques.
The paper tackles the challenge of representing dynamic graph topologies in multi-agent reinforcement learning environments, proposing a method that combines graph attention networks with Gumbel Sinkhorn networks to learn latent permutations, and it empirically outperforms existing methods in the PettingZoo environment.
Vital importance has necessity to be attached to cooperation in multi-agent environments, as a result of which some reinforcement learning algorithms combined with graph neural networks have been proposed to understand the mutual interplay between agents. However, highly complicated and dynamic multi-agent environments require more ingenious graph neural networks, which can comprehensively represent not only the graph topology structure but also evolution process of the structure due to agents emerging, disappearing and moving. To tackle these difficulties, we propose Gumbel Sinkhorn graph attention reinforcement learning, where a graph attention network highly represents the underlying graph topology structure of the multi-agent environment, and can adapt to the dynamic topology structure of graph better with the help of Gumbel Sinkhorn network by learning latent permutations. Empirically, simulation results show how our proposed graph reinforcement learning methodology outperforms existing methods in the PettingZoo multi-agent environment by learning latent permutations.