Towards Expressive Priors for Bayesian Neural Networks: Poisson Process Radial Basis Function Networks
This work addresses a limitation in Bayesian neural networks for researchers and practitioners by enabling more expressive priors, though it appears incremental as it builds on existing prior methods.
The authors tackled the problem of specifying basic properties like expected lengthscale and amplitude variance in Bayesian neural network priors by introducing Poisson Process Radial Basis Function Networks, which encode amplitude stationarity and input-dependent lengthscale, and they proved consistency as observations increase.
While Bayesian neural networks have many appealing characteristics, current priors do not easily allow users to specify basic properties such as expected lengthscale or amplitude variance. In this work, we introduce Poisson Process Radial Basis Function Networks, a novel prior that is able to encode amplitude stationarity and input-dependent lengthscale. We prove that our novel formulation allows for a decoupled specification of these properties, and that the estimated regression function is consistent as the number of observations tends to infinity. We demonstrate its behavior on synthetic and real examples.