Learning Representations in Video Game Agents with Supervised Contrastive Imitation Learning
This work addresses the challenge of capturing action-relevant factors in observations for video game agents, offering incremental improvements in representation learning for imitation learning tasks.
The paper tackles the problem of learning effective state representations for video game agents by applying Supervised Contrastive Learning to Imitation Learning, resulting in improved representation quality, faster convergence, and better generalization in 3D and 2D game environments compared to baseline models.
This paper introduces a novel application of Supervised Contrastive Learning (SupCon) to Imitation Learning (IL), with a focus on learning more effective state representations for agents in video game environments. The goal is to obtain latent representations of the observations that capture better the action-relevant factors, thereby modeling better the cause-effect relationship from the observations that are mapped to the actions performed by the demonstrator, for example, the player jumps whenever an obstacle appears ahead. We propose an approach to integrate the SupCon loss with continuous output spaces, enabling SupCon to operate without constraints regarding the type of actions of the environment. Experiments on the 3D games Astro Bot and Returnal, and multiple 2D Atari games show improved representation quality, faster learning convergence, and better generalization compared to baseline models trained only with supervised action prediction loss functions.