Simon Cotter

Colin Cotter, Simon Cotter, Paul Russell

Markov chain Monte Carlo methods are a powerful and commonly used family of numerical methods for sampling from complex probability distributions. As applications of these methods increase in size and complexity, the need for efficient methods increases. In this paper, we present a particle ensemble algorithm. At each iteration, an importance sampling proposal distribution is formed using an ensemble of particles. A stratified sample is taken from this distribution and weighted under the posterior, a state-of-the-art ensemble transport resampling method is then used to create an evenly weighted sample ready for the next iteration. We demonstrate that this ensemble transport adaptive importance sampling (ETAIS) method outperforms MCMC methods with equivalent proposal distributions for low dimensional problems, and in fact shows better than linear improvements in convergence rates with respect to the number of ensemble members. We also introduce a new resampling strategy, multinomial transformation (MT), which while not as accurate as the ensemble transport resampler, is substantially less costly for large ensemble sizes, and can then be used in conjunction with ETAIS for complex problems. We also focus on how algorithmic parameters regarding the mixture proposal can be quickly tuned to optimise performance. In particular, we demonstrate this methodology's superior sampling for multimodal problems, such as those arising from inference for mixture models, and for problems with expensive likelihoods requiring the solution of a differential equation, for which speed-ups of orders of magnitude are demonstrated. Likelihood evaluations of the ensemble could be computed in a distributed manner, suggesting that this methodology is a good candidate for parallel Bayesian computations.

James Rynn, Simon Cotter, Catherine E Powell et al.

Determination of the thermal properties of a material is an important task in many scientific and engineering applications. How a material behaves when subjected to high or fluctuating temperatures can be critical to the safety and longevity of a system's essential components. The laser flash experiment is a well-established technique for indirectly measuring the thermal diffusivity, and hence the thermal conductivity, of a material. In previous works, optimization schemes have been used to find estimates of the thermal conductivity and other quantities of interest which best fit a given model to experimental data. Adopting a Bayesian approach allows for prior beliefs about uncertain model inputs to be conditioned on experimental data to determine a posterior distribution, but probing this distribution using sampling techniques such as Markov chain Monte Carlo methods can be incredibly computationally intensive. This difficulty is especially true for forward models consisting of time-dependent partial differential equations. We pose the problem of determining the thermal conductivity of a material via the laser flash experiment as a Bayesian inverse problem in which the laser intensity is also treated as uncertain. We introduce a parametric surrogate model that takes the form of a stochastic Galerkin finite element approximation, also known as a generalized polynomial chaos expansion, and show how it can be used to sample efficiently from the approximate posterior distribution. This approach gives access not only to the sought-after estimate of the thermal conductivity but also important information about its relationship to the laser intensity, and information for uncertainty quantification. We also investigate the effects of the spatial profile of the laser on the estimated posterior distribution for the thermal conductivity.

Simon Cotter

Efficient analysis and simulation of multiscale stochastic systems of chemical kinetics is an ongoing area for research, and is the source of many theoretical and computational challenges. In this paper, we present a significant improvement to the constrained approach, which is a method for computing effective dynamics of slowly changing quantities in these systems, but which does not rely on the quasi-steady-state assumption (QSSA). The QSSA can cause errors in the estimation of effective dynamics for systems where the difference in timescales between the "fast" and "slow" variables is not so pronounced. This new application of the constrained approach allows us to compute the effective generator of the slow variables, without the need for expensive stochastic simulations. This is achieved by finding the null space of the generator of the constrained system. For complex systems where this is not possible, or where the constrained subsystem is itself multiscale, the constrained approach can then be applied iteratively. This results in breaking the problem down into finding the solutions to many small eigenvalue problems, which can be efficiently solved using standard methods. Since this methodology does not rely on the quasi steady-state assumption, the effective dynamics that are approximated are highly accurate, and in the case of systems with only monomolecular reactions, are exact. We will demonstrate this with some numerics, and also use the effective generators to sample paths of the slow variables which are conditioned on their endpoints, a task which would be computationally intractable for the generator of the full system.

Takuo Matsubara, Andrew Duncan, Simon Cotter et al.

We introduce bandit importance sampling (BIS), a new class of importance sampling methods designed for settings where the target density is expensive to evaluate. In contrast to adaptive importance sampling, which optimises a proposal distribution, BIS directly designs the samples through a sequential strategy that combines space-filling designs with multi-armed bandits. Our method leverages Gaussian process surrogates to guide sample selection, enabling efficient exploration of the parameter space with minimal target evaluations. We establish theoretical guarantees on convergence and demonstrate the effectiveness of the method across a broad range of sampling tasks. BIS delivers accurate approximations with fewer target evaluations, outperforming competing approaches across multimodal, heavy-tailed distributions, and real-world applications to Bayesian inference of computationally expensive models.

Simon Cotter, Radek Erban

Several different methods exist for efficient approximation of paths in multiscale stochastic chemical systems. Another approach is to use bursts of stochastic simulation to estimate the parameters of a stochastic differential equation approximation of the paths. In this paper, multiscale methods for approximating paths are used to formulate different strategies for estimating the dynamics by diffusion processes. We then analyse how efficient and accurate these methods are in a range of different scenarios, and compare their respective advantages and disadvantages to other methods proposed to analyse multiscale chemical networks.