Ergodic Generative Flows
This work addresses specific bottlenecks in GFNs for researchers in generative modeling and imitation learning, but it appears incremental as it builds on existing GFN frameworks.
The paper tackled challenges in training Generative Flow Networks (GFNs) for continuous settings and imitation learning by proposing Ergodic Generative Flows (EGFs), which use ergodicity to enable tractable flow-matching loss and a new KL-weakFM loss for imitation learning without a separate reward model, achieving results on toy tasks and real-world datasets from NASA.
Generative Flow Networks (GFNs) were initially introduced on directed acyclic graphs to sample from an unnormalized distribution density. Recent works have extended the theoretical framework for generative methods allowing more flexibility and enhancing application range. However, many challenges remain in training GFNs in continuous settings and for imitation learning (IL), including intractability of flow-matching loss, limited tests of non-acyclic training, and the need for a separate reward model in imitation learning. The present work proposes a family of generative flows called Ergodic Generative Flows (EGFs) which are used to address the aforementioned issues. First, we leverage ergodicity to build simple generative flows with finitely many globally defined transformations (diffeomorphisms) with universality guarantees and tractable flow-matching loss (FM loss). Second, we introduce a new loss involving cross-entropy coupled to weak flow-matching control, coined KL-weakFM loss. It is designed for IL training without a separate reward model. We evaluate IL-EGFs on toy 2D tasks and real-world datasets from NASA on the sphere, using the KL-weakFM loss. Additionally, we conduct toy 2D reinforcement learning experiments with a target reward, using the FM loss.