Game State Learning via Game Scene Augmentation
This incremental improvement addresses the need for precise game state extraction for AI tasks like game-playing and testing, but is domain-specific to game environments.
The authors tackled the problem of inferring accurate game state information from high-dimensional pixel input by introducing GameCLR, a game scene augmentation technique that leverages the game-engine to synthesize controlled renderings, which outperformed the SimCLR baseline in tests on the CARLA driving simulator environment.
Having access to accurate game state information is of utmost importance for any artificial intelligence task including game-playing, testing, player modeling, and procedural content generation. Self-Supervised Learning (SSL) techniques have shown to be capable of inferring accurate game state information from the high-dimensional pixel input of game footage into compressed latent representations. Contrastive Learning is a popular SSL paradigm where the visual understanding of the game's images comes from contrasting dissimilar and similar game states defined by simple image augmentation methods. In this study, we introduce a new game scene augmentation technique -- named GameCLR -- that takes advantage of the game-engine to define and synthesize specific, highly-controlled renderings of different game states, thereby, boosting contrastive learning performance. We test our GameCLR technique on images of the CARLA driving simulator environment and compare it against the popular SimCLR baseline SSL method. Our results suggest that GameCLR can infer the game's state information from game footage more accurately compared to the baseline. Our proposed approach allows us to conduct game artificial intelligence research by directly utilizing screen pixels as input.