Deep Latent Competition: Learning to Race Using Visual Control Policies in Latent Space
This addresses the challenge of adversarial interactions in multi-agent settings like racing, offering a method to learn visual control policies more efficiently, though it is incremental as it builds on existing world model and self-play techniques.
The paper tackles the problem of learning competitive behaviors in multi-agent racing by introducing Deep Latent Competition (DLC), a reinforcement learning algorithm that uses self-play in a learned latent world model to reduce real-world sample costs and enable scalable planning from images, achieving effective performance on a novel racing benchmark.
Learning competitive behaviors in multi-agent settings such as racing requires long-term reasoning about potential adversarial interactions. This paper presents Deep Latent Competition (DLC), a novel reinforcement learning algorithm that learns competitive visual control policies through self-play in imagination. The DLC agent imagines multi-agent interaction sequences in the compact latent space of a learned world model that combines a joint transition function with opponent viewpoint prediction. Imagined self-play reduces costly sample generation in the real world, while the latent representation enables planning to scale gracefully with observation dimensionality. We demonstrate the effectiveness of our algorithm in learning competitive behaviors on a novel multi-agent racing benchmark that requires planning from image observations. Code and videos available at https://sites.google.com/view/deep-latent-competition.