Distributional GFlowNets with Quantile Flows
This work addresses the problem of handling risk uncertainty in GFlowNets for researchers and practitioners in probabilistic sampling and reinforcement learning, representing an incremental advancement by extending the framework with distributional methods.
The paper tackled the limitation of Generative Flow Networks (GFlowNets) in handling stochastic rewards by introducing a distributional paradigm that turns flow functions into distributions, using quantile functions to enable risk-sensitive policies. This approach achieved substantial improvements on existing benchmarks compared to prior methods, even in deterministic reward settings.
Generative Flow Networks (GFlowNets) are a new family of probabilistic samplers where an agent learns a stochastic policy for generating complex combinatorial structure through a series of decision-making steps. Despite being inspired from reinforcement learning, the current GFlowNet framework is relatively limited in its applicability and cannot handle stochasticity in the reward function. In this work, we adopt a distributional paradigm for GFlowNets, turning each flow function into a distribution, thus providing more informative learning signals during training. By parameterizing each edge flow through their quantile functions, our proposed \textit{quantile matching} GFlowNet learning algorithm is able to learn a risk-sensitive policy, an essential component for handling scenarios with risk uncertainty. Moreover, we find that the distributional approach can achieve substantial improvement on existing benchmarks compared to prior methods due to our enhanced training algorithm, even in settings with deterministic rewards.