Learning Task Embeddings for Teamwork Adaptation in Multi-Agent Reinforcement Learning
This addresses the challenge of task adaptation for multi-agent systems in reinforcement learning, but it is incremental as it builds on existing embedding and adaptation methods.
The paper tackles the problem of teamwork adaptation in multi-agent reinforcement learning, where agents need to adapt to novel tasks with limited fine-tuning, and proposes multi-agent task embeddings (MATE) to enable this adaptation, showing that agents can adapt to novel tasks using these embeddings.
Successful deployment of multi-agent reinforcement learning often requires agents to adapt their behaviour. In this work, we discuss the problem of teamwork adaptation in which a team of agents needs to adapt their policies to solve novel tasks with limited fine-tuning. Motivated by the intuition that agents need to be able to identify and distinguish tasks in order to adapt their behaviour to the current task, we propose to learn multi-agent task embeddings (MATE). These task embeddings are trained using an encoder-decoder architecture optimised for reconstruction of the transition and reward functions which uniquely identify tasks. We show that a team of agents is able to adapt to novel tasks when provided with task embeddings. We propose three MATE training paradigms: independent MATE, centralised MATE, and mixed MATE which vary in the information used for the task encoding. We show that the embeddings learned by MATE identify tasks and provide useful information which agents leverage during adaptation to novel tasks.