Counter-Strike Deathmatch with Large-Scale Behavioural Cloning
This is an incremental advance in imitation learning for first-person shooter games, addressing the challenge of limited on-policy data in real-time environments.
The paper tackles the problem of creating an AI agent that plays Counter-Strike: Global Offensive from pixel input without an API, using behavioral cloning on a large dataset of human gameplay to match the performance of the medium difficulty built-in AI in deathmatch mode.
This paper describes an AI agent that plays the popular first-person-shooter (FPS) video game `Counter-Strike; Global Offensive' (CSGO) from pixel input. The agent, a deep neural network, matches the performance of the medium difficulty built-in AI on the deathmatch game mode, whilst adopting a humanlike play style. Unlike much prior work in games, no API is available for CSGO, so algorithms must train and run in real-time. This limits the quantity of on-policy data that can be generated, precluding many reinforcement learning algorithms. Our solution uses behavioural cloning - training on a large noisy dataset scraped from human play on online servers (4 million frames, comparable in size to ImageNet), and a smaller dataset of high-quality expert demonstrations. This scale is an order of magnitude larger than prior work on imitation learning in FPS games.