CFR-MIX: Solving Imperfect Information Extensive-Form Games with Combinatorial Action Space
This addresses a bottleneck in multi-agent coordination games for AI and game theory researchers, though it is incremental as it builds on CFR with novel strategy representations and decomposition methods.
The authors tackled the inefficiency of solving imperfect-information extensive-form games with combinatorial action spaces, where existing algorithms like Counterfactual Regret Minimization (CFR) struggle due to exponential growth in joint action space, and proposed CFR-MIX, which significantly outperforms existing algorithms in experiments.
In many real-world scenarios, a team of agents coordinate with each other to compete against an opponent. The challenge of solving this type of game is that the team's joint action space grows exponentially with the number of agents, which results in the inefficiency of the existing algorithms, e.g., Counterfactual Regret Minimization (CFR). To address this problem, we propose a new framework of CFR: CFR-MIX. Firstly, we propose a new strategy representation that represents a joint action strategy using individual strategies of all agents and a consistency relationship to maintain the cooperation between agents. To compute the equilibrium with individual strategies under the CFR framework, we transform the consistency relationship between strategies to the consistency relationship between the cumulative regret values. Furthermore, we propose a novel decomposition method over cumulative regret values to guarantee the consistency relationship between the cumulative regret values. Finally, we introduce our new algorithm CFR-MIX which employs a mixing layer to estimate cumulative regret values of joint actions as a non-linear combination of cumulative regret values of individual actions. Experimental results show that CFR-MIX outperforms existing algorithms on various games significantly.