Unlocking the Potential of Deep Counterfactual Value Networks
This work addresses the problem of enhancing AI performance in poker for researchers and developers, representing an incremental advancement over prior methods.
The paper tackled the limited adoption of deep counterfactual value networks in imperfect-information games by introducing improvements to them and counterfactual regret minimization, resulting in the poker AI Supremus beating the benchmark agent Slumbot by an extremely large margin and achieving lower exploitability than DeepStack.
Deep counterfactual value networks combined with continual resolving provide a way to conduct depth-limited search in imperfect-information games. However, since their introduction in the DeepStack poker AI, deep counterfactual value networks have not seen widespread adoption. In this paper we introduce several improvements to deep counterfactual value networks, as well as counterfactual regret minimization, and analyze the effects of each change. We combined these improvements to create the poker AI Supremus. We show that while a reimplementation of DeepStack loses head-to-head against the strong benchmark agent Slumbot, Supremus successfully beats Slumbot by an extremely large margin and also achieves a lower exploitability than DeepStack against a local best response. Together, these results show that with our key improvements, deep counterfactual value networks can achieve state-of-the-art performance.