Variational Bayesian strategies for high-dimensional, stochastic design problems
This addresses a lesser-studied challenge in uncertainty quantification for researchers and practitioners dealing with high-dimensional design problems, though it appears incremental as it builds on existing variational Bayesian methods.
The paper tackles the problem of optimization under uncertainty in high-dimensional stochastic design by proposing a Variational Bayesian framework that recasts it as probabilistic inference, achieving computational costs of O(10^2) forward model calls for problems with O(10^3) variables.
This paper is concerned with a lesser-studied problem in the context of model-based, uncertainty quantification (UQ), that of optimization/design/control under uncertainty. The solution of such problems is hindered not only by the usual difficulties encountered in UQ tasks (e.g. the high computational cost of each forward simulation, the large number of random variables) but also by the need to solve a nonlinear optimization problem involving large numbers of design variables and potentially constraints. We propose a framework that is suitable for a large class of such problems and is based on the idea of recasting them as probabilistic inference tasks. To that end, we propose a Variational Bayesian (VB) formulation and an iterative VB-Expectation-Maximization scheme that is also capable of identifying a low-dimensional set of directions in the design space, along which, the objective exhibits the largest sensitivity. We demonstrate the validity of the proposed approach in the context of two numerical examples involving $\mathcal{O}(10^3)$ random and design variables. In all cases considered the cost of the computations in terms of calls to the forward model was of the order $\mathcal{O}(10^2)$. The accuracy of the approximations provided is assessed by appropriate information-theoretic metrics.