Student-t processes as infinite-width limits of posterior Bayesian neural networks
This work offers a theoretical foundation for using Student-t processes as approximations in Bayesian deep learning, though it appears incremental as it extends existing Gaussian process results.
The authors tackled the problem of approximating posterior Bayesian neural networks in the infinite-width limit, showing they converge to Student-t processes rather than Gaussian processes, which provides greater flexibility in modeling uncertainty.
The asymptotic properties of Bayesian Neural Networks (BNNs) have been extensively studied, particularly regarding their approximations by Gaussian processes in the infinite-width limit. We extend these results by showing that posterior BNNs can be approximated by Student-t processes, which offer greater flexibility in modeling uncertainty. Specifically, we show that, if the parameters of a BNN follow a Gaussian prior distribution, and the variance of both the last hidden layer and the Gaussian likelihood function follows an Inverse-Gamma prior distribution, then the resulting posterior BNN converges to a Student-t process in the infinite-width limit. Our proof leverages the Wasserstein metric to establish control over the convergence rate of the Student-t process approximation.