Boundary Variance Inflation Causes Acquisition Bias in Gaussian Processes
For practitioners using Gaussian processes in Bayesian optimization, this reveals a fundamental flaw in acquisition behavior that can mislead optimization, especially in high-dimensional bounded domains.
The paper identifies that boundary-induced variance inflation in Gaussian processes causes acquisition bias in Bayesian optimization, independent of the objective function. It shows that this distortion worsens with dimensionality and manifests differently across acquisition classes, dominating task-specific uncertainty.
Gaussian processes with stationary kernels on bounded domains exhibit inflated posterior variance near the boundary. Despite being a long-recognized artifact in geostatistics and a source of over-exploration in Bayesian optimization, the causes and effects of boundary-induced acquisition bias are underexplored. We trace the root cause to a simple geometric mechanism: the truncation of the kernel correlation neighborhood at the domain boundary creates an observation-independent distortion that worsens with dimensionality. We show how this distortion manifests across three acquisition classes: variance maximization concentrates selections at the corners, whereas negative integrated posterior variance and expected predictive information gain move selections inward to axis-aligned interior shells. These patterns arise without reference to any objective function, meaning that acquisition behavior can be dominated by kernel geometry rather than the desired task-specific uncertainty. To quantify this, we introduce a function-free selection-profile diagnostic for arbitrary acquisitions, kernels, and bounded-domain geometries.