Sampling from Gaussian Processes: A Tutorial and Applications in Global Sensitivity Analysis and Optimization
It addresses efficiency issues in engineering optimization by introducing sampling techniques, but it is incremental as it adapts existing methods to a new domain.
The paper tackles the high cost of simulations in engineering by applying Gaussian process sampling methods for global sensitivity analysis and optimization, demonstrating their effectiveness through numerical examples.
High-fidelity simulations and physical experiments are essential for engineering analysis and design. However, their high cost often limits their applications in two critical tasks: global sensitivity analysis (GSA) and optimization. This limitation motivates the common use of Gaussian processes (GPs) as proxy regression models to provide uncertainty-aware predictions based on a limited number of high-quality observations. GPs naturally enable efficient sampling strategies that support informed decision-making under uncertainty by extracting information from a subset of possible functions for the model of interest. Despite their popularity in machine learning and statistics communities, sampling from GPs has received little attention in the community of engineering optimization. In this paper, we present the formulation and detailed implementation of two notable sampling methods -- random Fourier features and pathwise conditioning -- for generating posterior samples from GPs. Alternative approaches are briefly described. Importantly, we detail how the generated samples can be applied in GSA, single-objective optimization, and multi-objective optimization. We show successful applications of these sampling methods through a series of numerical examples.