Playing Catan with Cross-dimensional Neural Network
This addresses the problem of creating effective AI for complex strategic games with mixed information sources, though it appears incremental as it builds on existing RL approaches.
The researchers tackled the challenge of building AI agents for the complex board game Catan using reinforcement learning without domain knowledge, and demonstrated that their cross-dimensional neural network dramatically improves RL performance, enabling it to outperform the best heuristic agent jsettler for the first time.
Catan is a strategic board game having interesting properties, including multi-player, imperfect information, stochastic, complex state space structure (hexagonal board where each vertex, edge and face has its own features, cards for each player, etc), and a large action space (including negotiation). Therefore, it is challenging to build AI agents by Reinforcement Learning (RL for short), without domain knowledge nor heuristics. In this paper, we introduce cross-dimensional neural networks to handle a mixture of information sources and a wide variety of outputs, and empirically demonstrate that the network dramatically improves RL in Catan. We also show that, for the first time, a RL agent can outperform jsettler, the best heuristic agent available.