RLCFR: Minimize Counterfactual Regret by Deep Reinforcement Learning
This work addresses generalization in decision-making for imperfect-information games, which is an incremental improvement over existing CFR methods.
The authors tackled the problem of improving the generalization ability of Counterfactual Regret Minimization (CFR) in two-player zero-sum games with imperfect information by proposing RLCFR, a framework that integrates CFR into a reinforcement learning setup to learn a policy for regret updating, resulting in significantly improved generalization compared to state-of-the-art methods.
Counterfactual regret minimization (CFR) is a popular method to deal with decision-making problems of two-player zero-sum games with imperfect information. Unlike existing studies that mostly explore for solving larger scale problems or accelerating solution efficiency, we propose a framework, RLCFR, which aims at improving the generalization ability of the CFR method. In the RLCFR, the game strategy is solved by the CFR in a reinforcement learning framework. And the dynamic procedure of iterative interactive strategy updating is modeled as a Markov decision process (MDP). Our method, RLCFR, then learns a policy to select the appropriate way of regret updating in the process of iteration. In addition, a stepwise reward function is formulated to learn the action policy, which is proportional to how well the iteration strategy is at each step. Extensive experimental results on various games have shown that the generalization ability of our method is significantly improved compared with existing state-of-the-art methods.