Expected flow networks in stochastic environments and two-player zero-sum games
This work addresses the challenge of applying flow networks to stochastic and game-based settings, offering incremental improvements for domains like protein design and game AI.
The paper tackles the problem of extending generative flow networks to stochastic and adversarial environments, proposing expected flow networks (EFlowNets) and adversarial flow networks (AFlowNets), with results showing EFlowNets outperform other formulations in tasks like protein design and AFlowNets achieve over 80% optimal moves in Connect-4 and outperform AlphaZero.
Generative flow networks (GFlowNets) are sequential sampling models trained to match a given distribution. GFlowNets have been successfully applied to various structured object generation tasks, sampling a diverse set of high-reward objects quickly. We propose expected flow networks (EFlowNets), which extend GFlowNets to stochastic environments. We show that EFlowNets outperform other GFlowNet formulations in stochastic tasks such as protein design. We then extend the concept of EFlowNets to adversarial environments, proposing adversarial flow networks (AFlowNets) for two-player zero-sum games. We show that AFlowNets learn to find above 80% of optimal moves in Connect-4 via self-play and outperform AlphaZero in tournaments.