OpenHoldem: A Benchmark for Large-Scale Imperfect-Information Game Research
This work addresses the lack of standard benchmarks for researchers studying superhuman AIs in No-limit Texas Hold'em, which is an incremental improvement for the imperfect-information game research community.
This paper introduces OpenHoldem, a toolkit for large-scale imperfect-information game research using No-limit Texas Hold'em (NLTH). It provides a standardized evaluation protocol, four strong baselines, and an online testing platform to facilitate research in this area.
Owning to the unremitting efforts by a few institutes, significant progress has recently been made in designing superhuman AIs in No-limit Texas Hold'em (NLTH), the primary testbed for large-scale imperfect-information game research. However, it remains challenging for new researchers to study this problem since there are no standard benchmarks for comparing with existing methods, which seriously hinders further developments in this research area. In this work, we present OpenHoldem, an integrated toolkit for large-scale imperfect-information game research using NLTH. OpenHoldem makes three main contributions to this research direction: 1) a standardized evaluation protocol for thoroughly evaluating different NLTH AIs, 2) four publicly available strong baselines for NLTH AI, and 3) an online testing platform with easy-to-use APIs for public NLTH AI evaluation. We have released OpenHoldem at holdem.ia.ac.cn, hoping it facilitates further studies on the unsolved theoretical and computational issues in this area and cultivate crucial research problems like opponent modeling and human-computer interactive learning.