Robert Scheichl

Lukas Holbach, Peter Bastian, Robert Scheichl

We introduce a two-level hybrid restricted additive Schwarz (RAS) preconditioner for heterogeneous steady-state convection-diffusion equations at high Péclet numbers. Our construction builds on the multiscale spectral generalized finite element method (MS-GFEM), wherein the coarse space is spanned by locally optimal basis functions obtained from local generalized eigenproblems on operator-harmonic spaces. Extending the theory of Ma (2025) to convection-diffusion problems in conservation form, we establish exponential convergence of the MS-GFEM approximation with respect to the dimension of the local approximation space. Rewriting MS-GFEM as a RAS-type iteration, we show for coercive problems that this exponential convergence property is inherited by the RAS-type iterative method (at least in the continuous setting). Employed as a preconditioner within the generalized minimal residual method (GMRES), the resulting method requires only a few iterations for high accuracy even with low-dimensional coarse spaces. Through extensive numerical experiments on problems with high-contrast diffusion and non-divergence-free, rotating velocity fields, we demonstrate robustness with respect to the grid Péclet number and the number of subdomains (tested up to $10^5$ subdomains), while coarse-space dimensions remain small as grid Péclet numbers increase. By adapting the coarse space and oversampling size, we are able to achieve arbitrarily fast convergence of preconditioned GMRES. As an extension, for which we do not have theory yet, we show effectiveness of the method even for indefinite problems and in the vanishing-diffusion limit.

Tiangang Cui, Karina Koval, Roland Herzog et al.

We propose a novel approach for sequential optimal experimental design (sOED) for Bayesian inverse problems involving expensive models with high-dimensional unknown parameters. This work focuses on designs that maximize the expected information gain (EIG) from prior to posterior, a task that is computationally very challenging in non-Gaussian settings. This challenge is amplified in sOED, as the incremental expected information gain (iEIG) must be repeatedly approximated across distinct stages, with both prior and posterior distributions being intractable. To address this, we derive a general-purpose, derivative-based upper bound for the iEIG, which not only guides design placement but also enables the construction of projectors onto likelihood-informed subspaces, facilitating parameter dimension reduction. By combining this approach with conditional measure transport maps for the sequence of posteriors, we develop a unified sOED and amortized inference framework scalable to high- and infinite-dimensional problems. Numerical experiments for two inverse problems governed by partial differential equations (PDEs) demonstrate the effectiveness of designs by maximizing the proposed bound.

Sergey Dolgov, Robert Scheichl

We consider the approximate solution of parametric PDEs using the low-rank Tensor Train (TT) decomposition. Such parametric PDEs arise for example in uncertainty quantification problems in engineering applications. We propose an algorithm that is a hybrid of the alternating least squares and the TT cross methods. It computes a TT approximation of the whole solution, which is beneficial when multiple quantities of interest are sought. This might be needed, for example, for the computation of the probability density function (PDF) via the maximum entropy method [Kavehrad and Joseph, IEEE Trans. Comm., 1986]. The new algorithm exploits and preserves the block diagonal structure of the discretized operator in stochastic collocation schemes. This disentangles computations of the spatial and parametric degrees of freedom in the TT representation. In particular, it only requires solving independent PDEs at a few parameter values, thus allowing the use of existing high performance PDE solvers. In our numerical experiments, we apply the new algorithm to the stochastic diffusion equation and compare it with preconditioned steepest descent in the TT format, as well as with (multilevel) quasi-Monte Carlo and dimension-adaptive sparse grids methods. For sufficiently smooth random fields the new approach is orders of magnitude faster.

Eike Mueller, Xu Guo, Robert Scheichl et al.

Many problems in geophysical and atmospheric modelling require the fast solution of elliptic partial differential equations (PDEs) in "flat" three dimensional geometries. In particular, an anisotropic elliptic PDE for the pressure correction has to be solved at every time step in the dynamical core of many numerical weather prediction models, and equations of a very similar structure arise in global ocean models, subsurface flow simulations and gas and oil reservoir modelling. The elliptic solve is often the bottleneck of the forecast, and an algorithmically optimal method has to be used and implemented efficiently. Graphics Processing Units have been shown to be highly efficient for a wide range of applications in scientific computing, and recently iterative solvers have been parallelised on these architectures. We describe the GPU implementation and optimisation of a Preconditioned Conjugate Gradient (PCG) algorithm for the solution of a three dimensional anisotropic elliptic PDE for the pressure correction in NWP. Our implementation exploits the strong vertical anisotropy of the elliptic operator in the construction of a suitable preconditioner. As the algorithm is memory bound, performance can be improved significantly by reducing the amount of global memory access. We achieve this by using a matrix-free implementation which does not require explicit storage of the matrix and instead recalculates the local stencil. Global memory access can also be reduced by rewriting the algorithm using loop fusion and we show that this further reduces the runtime on the GPU. We demonstrate the performance of our matrix-free GPU code by comparing it to a sequential CPU implementation and to a matrix-explicit GPU code which uses existing libraries. The absolute performance of the algorithm for different problem sizes is quantified in terms of floating point throughput and global memory bandwidth.

Daniel Peterseim, Robert Scheichl

We present a new approach to the numerical upscaling for elliptic problems with rough diffusion coefficient at high contrast. It is based on the localizable orthogonal decomposition of $H^1$ into the image and the kernel of some novel stable quasi-interpolation operators with local $L^2$-approximation properties, independent of the contrast. We identify a set of sufficient assumptions on these quasi-interpolation operators that guarantee in principle optimal convergence without pre-asymptotic effects for high-contrast coefficients. We then give an example of a suitable operator and establish the assumptions for a particular class of high-contrast coefficients. So far this is not possible without any pre-asymptotic effects, but the optimal convergence is independent of the contrast and the asymptotic range is largely improved over other discretisation schemes. The new framework is sufficiently flexible to allow also for other choices of quasi-interpolation operators and the potential for fully robust numerical upscaling at high contrast.

Grigoris Katsiolides, Eike H. Müller, Robert Scheichl et al.

A common way to simulate the transport and spread of pollutants in the atmosphere is via stochastic Lagrangian dispersion models. Mathematically, these models describe turbulent transport processes with stochastic differential equations (SDEs). The computational bottleneck is the Monte Carlo algorithm, which simulates the motion of a large number of model particles in a turbulent velocity field; for each particle, a trajectory is calculated with a numerical timestepping method. Choosing an efficient numerical method is particularly important in operational emergency-response applications, such as tracking radioactive clouds from nuclear accidents or predicting the impact of volcanic ash clouds on international aviation, where accurate and timely predictions are essential. In this paper, we investigate the application of the Multilevel Monte Carlo (MLMC) method to simulate the propagation of particles in a representative one-dimensional dispersion scenario in the atmospheric boundary layer. MLMC can be shown to result in asymptotically superior computational complexity and reduced computational cost when compared to the Standard Monte Carlo (StMC) method, which is currently used in atmospheric dispersion modelling. To reduce the absolute cost of the method also in the non-asymptotic regime, it is equally important to choose the best possible numerical timestepping method on each level. To investigate this, we also compare the standard symplectic Euler method, which is used in many operational models, with two improved timestepping algorithms based on SDE splitting methods.

Ivan G. Graham, Matthew J. Parkinson, Robert Scheichl

We describe modern variants of Monte Carlo methods for Uncertainty Quantification (UQ) of the Neutron Transport Equation, when it is approximated by the discrete ordinates method with diamond differencing. We focus on the mono-energetic 1D slab geometry problem, with isotropic scattering, where the cross-sections are log-normal correlated random fields of possibly low regularity. The paper includes an outline of novel theoretical results on the convergence of the discrete scheme, in the cases of both spatially variable and random cross-sections. We also describe the theory and practice of algorithms for quantifying the uncertainty of a linear functional of the scalar flux, using Monte Carlo and quasi-Monte Carlo methods, and their multilevel variants. A hybrid iterative/direct solver for computing each realisation of the functional is also presented. Numerical experiments show the effectiveness of the hybrid solver and the gains that are possible through quasi-Monte Carlo sampling and multilevel variance reduction. For the multilevel quasi-Monte Carlo method, we observe gains in the computational $\varepsilon$-cost of up to 2 orders of magnitude over the standard Monte Carlo method, and we explain this theoretically. Experiments on problems with up to several thousand stochastic dimensions are included.

Tiangang Cui, Sergey Dolgov, Robert Scheichl

We propose a deep importance sampling method that is suitable for estimating rare event probabilities in high-dimensional problems. We approximate the optimal importance distribution in a general importance sampling problem as the pushforward of a reference distribution under a composition of order-preserving transformations, in which each transformation is formed by a squared tensor-train decomposition. The squared tensor-train decomposition provides a scalable ansatz for building order-preserving high-dimensional transformations via density approximations. The use of composition of maps moving along a sequence of bridging densities alleviates the difficulty of directly approximating concentrated density functions. To compute expectations over unnormalized probability distributions, we design a ratio estimator that estimates the normalizing constant using a separate importance distribution, again constructed via a composition of transformations in tensor-train format. This offers better theoretical variance reduction compared with self-normalized importance sampling, and thus opens the door to efficient computation of rare event probabilities in Bayesian inference problems. Numerical experiments on problems constrained by differential equations show little to no increase in the computational complexity with the event probability going to zero, and allow to compute hitherto unattainable estimates of rare event probabilities for complex, high-dimensional posterior densities.

Jakob Kruse, Gianluca Detommaso, Ullrich Köthe et al.

Many recent invertible neural architectures are based on coupling block designs where variables are divided in two subsets which serve as inputs of an easily invertible (usually affine) triangular transformation. While such a transformation is invertible, its Jacobian is very sparse and thus may lack expressiveness. This work presents a simple remedy by noting that subdivision and (affine) coupling can be repeated recursively within the resulting subsets, leading to an efficiently invertible block with dense, triangular Jacobian. By formulating our recursive coupling scheme via a hierarchical architecture, HINT allows sampling from a joint distribution p(y,x) and the corresponding posterior p(x|y) using a single invertible network. We evaluate our method on some standard data sets and benchmark its full power for density estimation and Bayesian inference on a novel data set of 2D shapes in Fourier parameterization, which enables consistent visualization of samples for different dimensionalities.

Alexander D. Gilbert, Ivan G. Graham, Frances Y. Kuo et al.

We consider the forward problem of uncertainty quantification for the generalised Dirichlet eigenvalue problem for a coercive second order partial differential operator with random coefficients, motivated by problems in structural mechanics, photonic crystals and neutron diffusion. The PDE coefficients are assumed to be uniformly bounded random fields, represented as infinite series parametrised by uniformly distributed i.i.d. random variables. The expectation of the fundamental eigenvalue of this problem is computed by (a) truncating the infinite series which define the coefficients; (b) approximating the resulting truncated problem using lowest order conforming finite elements and a sparse matrix eigenvalue solver; and (c) approximating the resulting finite (but high dimensional) integral by a randomly shifted quasi-Monte Carlo lattice rule, with specially chosen generating vector. We prove error estimates for the combined error, which depend on the truncation dimension $s$, the finite element mesh diameter $h$, and the number of quasi-Monte Carlo samples $N$. Under suitable regularity assumptions, our bounds are of the particular form $\mathcal{O}(h^2+N^{-1+δ})$, where $δ>0$ is arbitrary and the hidden constant is independent of the truncation dimension, which needs to grow as $h\to 0$ and $N\to\infty$. Although the eigenvalue problem is nonlinear, which means it is generally considered harder than the analogous source problem, in almost all cases we obtain error bounds that converge at the same rate as the corresponding rate for the source problem. The proof involves a detailed study of the regularity of the fundamental eigenvalue as a function of the random parameters. As a key intermediate result in the analysis, we prove that the spectral gap (between the fundamental and the second eigenvalues) is uniformly positive over all realisations of the random problem.

Gianluca Detommaso, Tiangang Cui, Alessio Spantini et al.

Stein variational gradient descent (SVGD) was recently proposed as a general purpose nonparametric variational inference algorithm [Liu & Wang, NIPS 2016]: it minimizes the Kullback-Leibler divergence between the target distribution and its approximation by implementing a form of functional gradient descent on a reproducing kernel Hilbert space. In this paper, we accelerate and generalize the SVGD algorithm by including second-order information, thereby approximating a Newton-like iteration in function space. We also show how second-order information can lead to more effective choices of kernel. We observe significant computational gains over the original SVGD algorithm in multiple test cases.

Anne Reinarz, Tim Dodwell, Tim Fletcher et al.

Finite element (FE) analysis has the potential to offset much of the expensive experimental testing currently required to certify aerospace laminates. However, large numbers of degrees of freedom are necessary to model entire aircraft components whilst accurately resolving micro-scale defects. The new module dune-composites, implemented within DUNE by the authors, provides a tool to efficiently solve large-scale problems using novel iterative solvers. The key innovation is a preconditioner that guarantees a constant number of iterations regardless of the problem size. Its robustness has been shown rigorously in Spillane et al. (Numer. Math. 126, 2014) for isotropic problems. For anisotropic problems in composites it is verified numerically for the first time in this paper. The parallel implementation in DUNE scales almost optimally over thousands of cores. To demonstrate this, we present an original numerical study, varying the shape of a localised wrinkle and the effect this has on the strength of a curved laminate. This requires a high-fidelity mesh containing at least four layers of quadratic elements across each ply and interface layer, underlining the need for dune-composites, which can achieve run times of just over 2 minutes on 2048 cores for realistic composites problems with 173 million degrees of freedom.

Frances Y. Kuo, Robert Scheichl, Christoph Schwab et al.

In this paper we present a rigorous cost and error analysis of a multilevel estimator based on randomly shifted Quasi-Monte Carlo (QMC) lattice rules for lognormal diffusion problems. These problems are motivated by uncertainty quantification problems in subsurface flow. We extend the convergence analysis in [Graham et al., Numer. Math. 2014] to multilevel Quasi-Monte Carlo finite element discretizations and give a constructive proof of the dimension-independent convergence of the QMC rules. More precisely, we provide suitable parameters for the construction of such rules that yield the required variance reduction for the multilevel scheme to achieve an $\varepsilon$-error with a cost of $\mathcal{O}(\varepsilon^{-θ})$ with $θ< 2$, and in practice even $θ\approx 1$, for sufficiently fast decaying covariance kernels of the underlying Gaussian random field inputs. This confirms that the computational gains due to the application of multilevel sampling methods and the gains due to the application of QMC methods, both demonstrated in earlier works for the same model problem, are complementary. A series of numerical experiments confirms these gains. The results show that in practice the multilevel QMC method consistently outperforms both the multilevel MC method and the single-level variants even for non-smooth problems.