Deep Surrogate Assisted MAP-Elites for Automated Hearthstone Deckbuilding
This work addresses the challenge of sample efficiency in automated content generation for games like Hearthstone, representing an incremental improvement over existing quality diversity methods.
The paper tackles the problem of efficiently generating high-quality and diverse decks in Hearthstone by proposing a deep surrogate model to assist MAP-Elites, reducing the number of expensive evaluations needed and setting a new state-of-the-art in automated deckbuilding.
We study the problem of efficiently generating high-quality and diverse content in games. Previous work on automated deckbuilding in Hearthstone shows that the quality diversity algorithm MAP-Elites can generate a collection of high-performing decks with diverse strategic gameplay. However, MAP-Elites requires a large number of expensive evaluations to discover a diverse collection of decks. We propose assisting MAP-Elites with a deep surrogate model trained online to predict game outcomes with respect to candidate decks. MAP-Elites discovers a diverse dataset to improve the surrogate model accuracy, while the surrogate model helps guide MAP-Elites towards promising new content. In a Hearthstone deckbuilding case study, we show that our approach improves the sample efficiency of MAP-Elites and outperforms a model trained offline with random decks, as well as a linear surrogate model baseline, setting a new state-of-the-art for quality diversity approaches in automated Hearthstone deckbuilding. We include the source code for all the experiments at: https://github.com/icaros-usc/EvoStone2.