StratFormer: Adaptive Opponent Modeling and Exploitation in Imperfect-Information Games
For AI in imperfect-information games, StratFormer provides a method to safely exploit opponents while maintaining near-equilibrium safety, though tested only on a small poker variant.
StratFormer is a transformer-based meta-agent that models and exploits opponents in imperfect-information games via a two-phase curriculum, achieving an average exploitation gain of +0.106 BB per hand over GTO in Leduc Hold'em, with peak gains of +0.821 against highly exploitable opponents.
We present StratFormer, a transformer-based meta-agent that learns to simultaneously model and exploit opponents in imperfect-information games through a two-phase curriculum. The first phase trains an opponent modeling head to identify behavioral patterns from action histories while the agent plays a game-theoretic optimal (GTO) policy. The second phase progressively shifts the policy toward best-response (BR) exploitation, guided by a per-opponent regularization schedule tied to exploitability. Our architecture introduces dual-turn tokens -- feature vectors constructed at both agent and opponent decision points -- coupled with bucket-rate features that encode opponent tendencies across five strategic contexts. On Leduc Hold'em, a small poker variant with six cards and two betting rounds, we test against six opponent archetypes at two strength levels each, with exploitability ranging from 0.15 to 1.26 Big Blinds (BB) per hand. StratFormer achieves an average exploitation gain of +0.106 BB per hand over GTO, with peak gains of +0.821 against highly exploitable opponents, while maintaining near-equilibrium safety.