Semantic Tracklets: An Object-Centric Representation for Visual Multi-Agent Reinforcement Learning
This addresses scalability issues in MARL for visual environments like autonomous fleet control, offering a novel representation approach.
The paper tackled the challenge of scaling multi-agent reinforcement learning (MARL) to visual inputs by introducing an object-centric intermediate representation called 'semantic tracklets', which outperformed baselines with a +2.4 higher score difference on GFootball and enabled learning for five players using only visual data.
Solving complex real-world tasks, e.g., autonomous fleet control, often involves a coordinated team of multiple agents which learn strategies from visual inputs via reinforcement learning. Many existing multi-agent reinforcement learning (MARL) algorithms however don't scale to environments where agents operate on visual inputs. To address this issue, algorithmically, recent works have focused on non-stationarity and exploration. In contrast, we study whether scalability can also be achieved via a disentangled representation. For this, we explicitly construct an object-centric intermediate representation to characterize the states of an environment, which we refer to as `semantic tracklets.' We evaluate `semantic tracklets' on the visual multi-agent particle environment (VMPE) and on the challenging visual multi-agent GFootball environment. `Semantic tracklets' consistently outperform baselines on VMPE, and achieve a +2.4 higher score difference than baselines on GFootball. Notably, this method is the first to successfully learn a strategy for five players in the GFootball environment using only visual data.