Representation Learning For Efficient Deep Multi-Agent Reinforcement Learning
This work addresses the problem of slow learning in MARL for researchers and practitioners, offering an incremental enhancement to existing methods.
The paper tackles the sample efficiency challenge in multi-agent reinforcement learning (MARL) by introducing MAPO-LSO, a method that integrates latent representation learning with MARL objectives, resulting in notable improvements in sample efficiency and learning performance across diverse tasks without extra hyperparameter tuning.
Sample efficiency remains a key challenge in multi-agent reinforcement learning (MARL). A promising approach is to learn a meaningful latent representation space through auxiliary learning objectives alongside the MARL objective to aid in learning a successful control policy. In our work, we present MAPO-LSO (Multi-Agent Policy Optimization with Latent Space Optimization) which applies a form of comprehensive representation learning devised to supplement MARL training. Specifically, MAPO-LSO proposes a multi-agent extension of transition dynamics reconstruction and self-predictive learning that constructs a latent state optimization scheme that can be trivially extended to current state-of-the-art MARL algorithms. Empirical results demonstrate MAPO-LSO to show notable improvements in sample efficiency and learning performance compared to its vanilla MARL counterpart without any additional MARL hyperparameter tuning on a diverse suite of MARL tasks.