Surrogate-based quantification of policy uncertainty in generative flow networks
This addresses uncertainty quantification for researchers using generative flow networks, but it is incremental as it builds on existing methods for uncertainty estimation.
The paper tackles the problem of epistemic uncertainty in generative flow networks due to approximate reward functions by introducing a surrogate model based on polynomial chaos expansion to quantify policy uncertainty, demonstrating performance on tasks like grid-world and symbolic regression.
Generative flow networks are able to sample, via sequential construction, high-reward, complex objects according to a reward function. However, such reward functions are often estimated approximately from noisy data, leading to epistemic uncertainty in the learnt policy. We present an approach to quantify this uncertainty by constructing a surrogate model composed of a polynomial chaos expansion, fit on a small ensemble of trained flow networks. This model learns the relationship between reward functions, parametrised in a low-dimensional space, and the probability distributions over actions at each step along a trajectory of the flow network. The surrogate model can then be used for inexpensive Monte Carlo sampling to estimate the uncertainty in the policy given uncertain rewards. We illustrate the performance of our approach on a discrete and continuous grid-world, symbolic regression, and a Bayesian structure learning task.