Marc Day

Matthias Morzfeld, Marcus S. Day, Ray W. Grout et al.

In parameter estimation problems one computes a posterior distribution over uncertain parameters defined jointly by a prior distribution, a model, and noisy data. Markov Chain Monte Carlo (MCMC) is often used for the numerical solution of such problems. An alternative to MCMC is importance sampling, which can exhibit near perfect scaling with the number of cores on high performance computing systems because samples are drawn independently. However, finding a suitable proposal distribution is a challenging task. Several sampling algorithms have been proposed over the past years that take an iterative approach to constructing a proposal distribution. We investigate the applicability of such algorithms by applying them to two realistic and challenging test problems, one in subsurface flow, and one in combustion modeling. More specifically, we implement importance sampling algorithms that iterate over the mean and covariance matrix of Gaussian or multivariate t-proposal distributions. Our implementation leverages massively parallel computers, and we present strategies to initialize the iterations using "coarse" MCMC runs or Gaussian mixture models.

John Bell, Marcus Day, Jonathan Goodman et al.

First-principles Markov Chain Monte Carlo sampling is used to investigate uncertainty quantification and uncertainty propagation in parameters describing hydrogen kinetics. Specifically, we sample the posterior distribution of thirty-one parameters focusing on the H2O2 and HO2 reactions resulting from conditioning on ninety-one experiments. Established literature values are used for the remaining parameters in the mechanism. The samples are computed using an affine invariant sampler starting with broad, noninformative priors. Autocorrelation analysis shows that O(1M) samples are sufficient to obtain a reasonable sampling of the posterior. The resulting distribution identifies strong positive and negative correlations and several non-Gaussian characteristics. Using samples drawn from the posterior, we investigate the impact of parameter uncertainty on the prediction of two more complex flames: a 2D premixed flame kernel and the ignition of a hydrogen jet issuing into a heated chamber. The former represents a combustion regime similar to the target experiments used to calibrate the mechanism and the latter represents a different combustion regime. For the premixed flame, the net amount of product after a given time interval has a standard deviation of less than 2% whereas the standard deviation of the ignition time for the jet is more than 10%. The samples used for these studies are posted online. These results indicate the degree to which parameters consistent with the target experiments constrain predicted behavior in different combustion regimes. This process provides a framework for both identifying reactions for further study from candidate mechanisms as well as combining uncertainty quantification and propagation to, ultimately, tie uncertainty in laboratory flame experiments to uncertainty in end-use numerical predictions of more complicated scenarios.

Vincent Dumont, Casey Garner, Anuradha Trivedi et al.

We present a new software, HYPPO, that enables the automatic tuning of hyperparameters of various deep learning (DL) models. Unlike other hyperparameter optimization (HPO) methods, HYPPO uses adaptive surrogate models and directly accounts for uncertainty in model predictions to find accurate and reliable models that make robust predictions. Using asynchronous nested parallelism, we are able to significantly alleviate the computational burden of training complex architectures and quantifying the uncertainty. HYPPO is implemented in Python and can be used with both TensorFlow and PyTorch libraries. We demonstrate various software features on time-series prediction and image classification problems as well as a scientific application in computed tomography image reconstruction. Finally, we show that (1) we can reduce by an order of magnitude the number of evaluations necessary to find the most optimal region in the hyperparameter space and (2) we can reduce by two orders of magnitude the throughput for such HPO process to complete.

Jaideep Pathak, Mustafa Mustafa, Karthik Kashinath et al.

Simulation of turbulent flows at high Reynolds number is a computationally challenging task relevant to a large number of engineering and scientific applications in diverse fields such as climate science, aerodynamics, and combustion. Turbulent flows are typically modeled by the Navier-Stokes equations. Direct Numerical Simulation (DNS) of the Navier-Stokes equations with sufficient numerical resolution to capture all the relevant scales of the turbulent motions can be prohibitively expensive. Simulation at lower-resolution on a coarse-grid introduces significant errors. We introduce a machine learning (ML) technique based on a deep neural network architecture that corrects the numerical errors induced by a coarse-grid simulation of turbulent flows at high-Reynolds numbers, while simultaneously recovering an estimate of the high-resolution fields. Our proposed simulation strategy is a hybrid ML-PDE solver that is capable of obtaining a meaningful high-resolution solution trajectory while solving the system PDE at a lower resolution. The approach has the potential to dramatically reduce the expense of turbulent flow simulations. As a proof-of-concept, we demonstrate our ML-PDE strategy on a two-dimensional turbulent (Rayleigh Number $Ra=10^9$) Rayleigh-Bénard Convection (RBC) problem.