VFunc: a Deep Generative Model for Functions
This work addresses Bayesian deep learning and reinforcement learning in function space, offering potential benefits over existing methods like bootstrapping and parameter noise, but it appears incremental as it builds on prior generative models.
The authors introduced VFunc, a deep generative model for functions that provides a joint distribution over functions and latent variables, enabling efficient sampling and entropy maximization. They demonstrated proof-of-concept experiments for regression and reinforcement learning.
We introduce a deep generative model for functions. Our model provides a joint distribution p(f, z) over functions f and latent variables z which lets us efficiently sample from the marginal p(f) and maximize a variational lower bound on the entropy H(f). We can thus maximize objectives of the form E_{f~p(f)}[R(f)] + c*H(f), where R(f) denotes, e.g., a data log-likelihood term or an expected reward. Such objectives encompass Bayesian deep learning in function space, rather than parameter space, and Bayesian deep RL with representations of uncertainty that offer benefits over bootstrapping and parameter noise. In this short paper we describe our model, situate it in the context of prior work, and present proof-of-concept experiments for regression and RL.