Bayesian Probabilistic Numerical Integration with Tree-Based Models
This work addresses the problem of numerical integration uncertainty quantification for researchers in statistics and machine learning, offering a novel method that is incremental in improving upon existing BQ approaches.
The paper tackles the limitations of Bayesian quadrature (BQ) in high-dimensional or non-smooth functions by proposing BART-Int, a new algorithm based on Bayesian Additive Regression Trees priors, which demonstrates explicit convergence rates and is tested on benchmarks like Genz functions and a Bayesian survey design problem.
Bayesian quadrature (BQ) is a method for solving numerical integration problems in a Bayesian manner, which allows users to quantify their uncertainty about the solution. The standard approach to BQ is based on a Gaussian process (GP) approximation of the integrand. As a result, BQ is inherently limited to cases where GP approximations can be done in an efficient manner, thus often prohibiting very high-dimensional or non-smooth target functions. This paper proposes to tackle this issue with a new Bayesian numerical integration algorithm based on Bayesian Additive Regression Trees (BART) priors, which we call BART-Int. BART priors are easy to tune and well-suited for discontinuous functions. We demonstrate that they also lend themselves naturally to a sequential design setting and that explicit convergence rates can be obtained in a variety of settings. The advantages and disadvantages of this new methodology are highlighted on a set of benchmark tests including the Genz functions, and on a Bayesian survey design problem.