Generative Flow Networks: a Markov Chain Perspective
This work offers a theoretical foundation for GFlowNets, potentially benefiting researchers in machine learning and statistics dealing with sampling challenges, but it appears incremental as it builds on existing connections without introducing new methods or applications.
The paper tackles the problem of slow convergence in Markov chain Monte Carlo methods for sampling from multi-modal distributions by formalizing Generative Flow Networks (GFlowNets) as recurrent Markov chains, providing a unifying theoretical framework that connects them to MCMC and highlights their differences.
While Markov chain Monte Carlo methods (MCMC) provide a general framework to sample from a probability distribution defined up to normalization, they often suffer from slow convergence to the target distribution when the latter is highly multi-modal. Recently, Generative Flow Networks (GFlowNets) have been proposed as an alternative framework to mitigate this issue when samples have a clear compositional structure, by treating sampling as a sequential decision making problem. Although they were initially introduced from the perspective of flow networks, the recent advances of GFlowNets draw more and more inspiration from the Markov chain literature, bypassing completely the need for flows. In this paper, we formalize this connection and offer a new perspective for GFlowNets using Markov chains, showing a unifying view for GFlowNets regardless of the nature of the state space as recurrent Markov chains. Positioning GFlowNets under the same theoretical framework as MCMC methods also allows us to identify the similarities between both frameworks, and most importantly to highlight their