Detecting and Adapting to Novelty in Games
This addresses the challenge for game-playing agents to handle unexpected rule changes, but it is incremental as it builds on existing model-based reinforcement learning techniques.
The paper tackles the problem of open-world novelty in games, where rules change abruptly, by proposing a model-based reinforcement learning approach that uses knowledge graphs to represent game state and rules, enabling novelty detection and quick adaptation through imagination-based re-training.
Open-world novelty occurs when the rules of an environment can change abruptly, such as when a game player encounters "house rules". To address open-world novelty, game playing agents must be able to detect when novelty is injected, and to quickly adapt to the new rules. We propose a model-based reinforcement learning approach where game state and rules are represented as knowledge graphs. The knowledge graph representation of the state and rules allows novelty to be detected as changes in the knowledge graph, assists with the training of deep reinforcement learners, and enables imagination-based re-training where the agent uses the knowledge graph to perform look-ahead.