Contrastive learning-based agent modeling for deep reinforcement learning
This addresses the challenge of designing adaptive policies for intelligent agents in multi-agent systems, offering a more practical approach by relying only on ego agent observations, though it appears incremental as it builds on existing contrastive learning and agent modeling techniques.
The paper tackled the problem of agent modeling in multi-agent systems by proposing a contrastive learning-based method that removes assumptions about needing local observations from other agents or long trajectories, achieving state-of-the-art performance in cooperative and competitive tasks.
Multi-agent systems often require agents to collaborate with or compete against other agents with diverse goals, behaviors, or strategies. Agent modeling is essential when designing adaptive policies for intelligent machine agents in multiagent systems, as this is the means by which the ego agent understands other agents' behavior and extracts their meaningful policy representations. These representations can be used to enhance the ego agent's adaptive policy which is trained by reinforcement learning. However, existing agent modeling approaches typically assume the availability of local observations from other agents (modeled agents) during training or a long observation trajectory for policy adaption. To remove these constrictive assumptions and improve agent modeling performance, we devised a Contrastive Learning-based Agent Modeling (CLAM) method that relies only on the local observations from the ego agent during training and execution. With these observations, CLAM is capable of generating consistent high-quality policy representations in real-time right from the beginning of each episode. We evaluated the efficacy of our approach in both cooperative and competitive multi-agent environments. Our experiments demonstrate that our approach achieves state-of-the-art on both cooperative and competitive tasks, highlighting the potential of contrastive learning-based agent modeling for enhancing reinforcement learning.