Playing Non-Embedded Card-Based Games with Reinforcement Learning
This addresses the challenge of creating fair, human-like AI agents for complex games, though it is incremental as it applies existing methods to a new domain.
The authors tackled the problem of developing non-embedded AI agents for card-based RTS games like Clash Royale, achieving real-time autonomous gameplay that successfully defeats built-in AI opponents using visual inputs and offline reinforcement learning.
Significant progress has been made in AI for games, including board games, MOBA, and RTS games. However, complex agents are typically developed in an embedded manner, directly accessing game state information, unlike human players who rely on noisy visual data, leading to unfair competition. Developing complex non-embedded agents remains challenging, especially in card-based RTS games with complex features and large state spaces. We propose a non-embedded offline reinforcement learning training strategy using visual inputs to achieve real-time autonomous gameplay in the RTS game Clash Royale. Due to the lack of a object detection dataset for this game, we designed an efficient generative object detection dataset for training. We extract features using state-of-the-art object detection and optical character recognition models. Our method enables real-time image acquisition, perception feature fusion, decision-making, and control on mobile devices, successfully defeating built-in AI opponents. All code is open-sourced at https://github.com/wty-yy/katacr.