Evolution Guided Generative Flow Networks
This addresses a specific bottleneck in GFlowNets training for researchers and practitioners in probabilistic generative modeling, though it appears incremental as an augmentation to existing methods.
The paper tackles the challenge of training Generative Flow Networks (GFlowNets) effectively with long time horizons and sparse rewards by proposing Evolution guided generative flow networks (EGFN), which augments training using Evolutionary algorithms to store trajectories in a replay buffer, resulting in demonstrated effectiveness across toy and real-world benchmarks.
Generative Flow Networks (GFlowNets) are a family of probabilistic generative models that learn to sample compositional objects proportional to their rewards. One big challenge of GFlowNets is training them effectively when dealing with long time horizons and sparse rewards. To address this, we propose Evolution guided generative flow networks (EGFN), a simple but powerful augmentation to the GFlowNets training using Evolutionary algorithms (EA). Our method can work on top of any GFlowNets training objective, by training a set of agent parameters using EA, storing the resulting trajectories in the prioritized replay buffer, and training the GFlowNets agent using the stored trajectories. We present a thorough investigation over a wide range of toy and real-world benchmark tasks showing the effectiveness of our method in handling long trajectories and sparse rewards. We release the code at http://github.com/zarifikram/egfn.