Benchmarking End-to-End Behavioural Cloning on Video Games
This work addresses the challenge of creating general AI agents for video games without game-specific engineering, but it is incremental as it highlights limitations in current approaches.
The study evaluated the general applicability of end-to-end behavioral cloning on twelve video games, including six modern ones, using human demonstrations as training data, and found that while agents learned basic dynamics and rules, they could not match human performance in raw metrics.
Behavioural cloning, where a computer is taught to perform a task based on demonstrations, has been successfully applied to various video games and robotics tasks, with and without reinforcement learning. This also includes end-to-end approaches, where a computer plays a video game like humans do: by looking at the image displayed on the screen, and sending keystrokes to the game. As a general approach to playing video games, this has many inviting properties: no need for specialized modifications to the game, no lengthy training sessions and the ability to re-use the same tools across different games. However, related work includes game-specific engineering to achieve the results. We take a step towards a general approach and study the general applicability of behavioural cloning on twelve video games, including six modern video games (published after 2010), by using human demonstrations as training data. Our results show that these agents cannot match humans in raw performance but do learn basic dynamics and rules. We also demonstrate how the quality of the data matters, and how recording data from humans is subject to a state-action mismatch, due to human reflexes.