A comparison of Markov Chain Monte Carlo algorithms for Bayesian inference of constitutive models
This work provides practical guidance for selecting MCMC samplers in engineering applications, but it is incremental as it compares existing methods without introducing new algorithms.
The paper compared three MCMC algorithms (Metropolis-Hastings, Affine Invariant Stretch Move, and No-U-Turn Sampler) for Bayesian inference in constitutive models, finding that NUTS was beneficial for a viscous flow system due to high effective sample size but not for a thermal conduction system with expensive model evaluations.
Employing Bayesian inference to calibrate constitutive model parameters has grown substantially in recent years. Among the available techniques, Markov Chain Monte Carlo (MCMC) sampling remains one of the most widely used approaches for estimating the posterior distribution. Nevertheless, the selection of a specific MCMC algorithm is often driven by practical considerations, such as software availability or prior user experience. To support sampler selection, we present a comparison of three prominent samplers in the context of two distinct physical systems: a thermal conduction system and a viscous flow system. Calibration data are obtained through tailor-made experimental setups. We use the Kullback-Leibler (KL) divergence, which quantifies the statistical distance between the sampled posterior and the reference ('true') posterior, as a measure of convergence to compare the performance of the following MCMC sampling methods: the Metropolis-Hastings (MH) sampler, the Affine Invariant Stretch Move (AISM) sampler, and the No-U-Turn Sampler (NUTS). We study how this metric correlates to heuristic indicators such as the Gelman-Rubin diagnostic and the effective sample size. In addition, we assess the samplers' computational effort in terms of required number of model evaluations. Based on the results, we find that the heuristic convergence and performance indicators provide a good qualitative measure for KL-divergence for both systems. Regarding computational effort, the NUTS is net beneficial for the viscous flow system, as the high effective sample size outweighs the additional effort required for gradient-based proposal generation. For the thermal conduction system, which involves more expensive model evaluations, the NUTS is not advantageous. Thus, the computational efficiency of gradient evaluations is an important argument in sampler selection.