MimicBot: Combining Imitation and Reinforcement Learning to win in Bot Bowl
This addresses the challenge of developing competitive AI for a specific game domain, Bot Bowl, where domain knowledge previously dominated, but it is incremental as it builds on hybrid methods rather than introducing a new paradigm.
The paper tackles the problem of creating an AI agent that can win in the Bot Bowl III competition for Fantasy Football AI, where previous machine learning approaches failed to beat scripted bots. The result is MimicBot, which combines imitation and reinforcement learning to consistently defeat scripted agents and win the competition, while also improving sample efficiency in training.
This paper describe an hybrid agent trained to play in Fantasy Football AI which participated in the Bot Bowl III competition. The agent, MimicBot, is implemented using a specifically designed deep policy network and trained using a combination of imitation and reinforcement learning. Previous attempts in using a reinforcement learning approach in such context failed for a number of reasons, e.g. due to the intrinsic randomness in the environment and the large and uneven number of actions available, with a curriculum learning approach failing to consistently beat a randomly paying agent. Currently no machine learning approach can beat a scripted bot which makes use of the domain knowledge on the game. Our solution, thanks to an imitation learning and a hybrid decision-making process, consistently beat such scripted agents. Moreover we shed lights on how to more efficiently train in a reinforcement learning setting while drastically increasing sample efficiency. MimicBot is the winner of the Bot Bowl III competition, and it is currently the state-of-the-art solution.