Yanlai Chen

Yanlai Chen, Sigal Gottlieb

In this paper, we present the first reduced basis method well-suited for the collocation framework. Two fundamentally different algorithms are presented: the so-called Least Squares Reduced Collocation Method (LSRCM) and Empirical Reduced Collocation Method (ERCM). This work provides a reduced basis strategy to practitioners who {prefer} a collocation, rather than Galerkin, approach. Furthermore, the empirical reduced collocation method eliminates a potentially costly online procedure that is needed for non-affine problems with Galerkin approach. Numerical results demonstrate the high efficiency and accuracy of the reduced collocation methods, which match or exceed that of the traditional reduced basis method in the Galerkin framework.

Yanlai Chen, Sigal Gottlieb, Alfa Heryudono et al.

We propose and test the first Reduced Radial Basis Function Method (R$^2$BFM) for solving parametric partial differential equations on irregular domains. The two major ingredients are a stable Radial Basis Function (RBF) solver that has an optimized set of centers chosen through a reduced-basis-type greedy algorithm, and a collocation-based model reduction approach that systematically generates a reduced-order approximation whose dimension is orders of magnitude smaller than the total number of RBF centers. The resulting algorithm is efficient and accurate as demonstrated through two- and three-dimensional test problems.

Yanlai Chen, Shawn Koohy

Physics-Informed Neural Network (PINN) has proven itself a powerful tool to obtain the numerical solutions of nonlinear partial differential equations (PDEs) leveraging the expressivity of deep neural networks and the computing power of modern heterogeneous hardware. However, its training is still time-consuming, especially in the multi-query and real-time simulation settings, and its parameterization often overly excessive. In this paper, we propose the Generative Pre-Trained PINN (GPT-PINN) to mitigate both challenges in the setting of parametric PDEs. GPT-PINN represents a brand-new meta-learning paradigm for parametric systems. As a network of networks, its outer-/meta-network is hyper-reduced with only one hidden layer having significantly reduced number of neurons. Moreover, its activation function at each hidden neuron is a (full) PINN pre-trained at a judiciously selected system configuration. The meta-network adaptively ``learns'' the parametric dependence of the system and ``grows'' this hidden layer one neuron at a time. In the end, by encompassing a very small number of networks trained at this set of adaptively-selected parameter values, the meta-network is capable of generating surrogate solutions for the parametric system across the entire parameter domain accurately and efficiently.

Ngoc-Cuong Nguyen, Yanlai Chen

We present a class of reduced basis (RB) methods for the iterative solution of parametrized symmetric positive-definite (SPD) linear systems. The essential ingredients are a Galerkin projection of the underlying parametrized system onto a reduced basis space to obtain a reduced system; an adaptive greedy algorithm to efficiently determine sampling parameters and associated basis vectors; an offline-online computational procedure and a multi-fidelity approach to decouple the construction and application phases of the reduced basis method; and solution procedures to employ the reduced basis approximation as a {\em stand-alone iterative solver} or as a {\em preconditioner} in the conjugate gradient method. We present numerical examples to demonstrate the performance of the proposed methods in comparison with multigrid methods. Numerical results show that, when applied to solve linear systems resulting from discretizing the Poisson's equations, the speed of convergence of our methods matches or surpasses that of the multigrid-preconditioned conjugate gradient method, while their computational cost per iteration is significantly smaller providing a feasible alternative when the multigrid approach is out of reach due to timing or memory constraints for large systems. Moreover, numerical results verify that this new class of reduced basis methods, when applied as a stand-alone solver or as a preconditioner, is capable of achieving the accuracy at the level of the {\em truth approximation} which is far beyond the RB level.

Lijie Ji, Yanlai Chen, Zhenli Xu

In numerical simulations of many charged systems at the micro/nano scale, a common theme is the repeated solution of the Poisson-Boltzmann equation. This task proves challenging, if not entirely infeasible, largely due to the nonlinearity of the equation and the high dimensionality of the physical and parametric domains with the latter emulating the system configuration. In this paper, we for the first time adapt a mathematically rigorous and computationally efficient model order reduction paradigm, the so-called reduced basis method (RBM), to mitigate this challenge. We adopt a finite difference method as the mandatory underlying scheme to produce the {\em truth approximations} of the RBM upon which the fast algorithm is built and its performance is measured against. Numerical tests presented in this paper demonstrate the high efficiency and accuracy of the fast algorithm, the reliability of its error estimation, as well as its capability in effectively capturing the boundary layer.

Yanlai Chen

We present a certified version of the Natural-Norm Successive Constraint Method (cNNSCM) for fast and accurate Inf-Sup lower bound evaluation of parametric operators. Successive Constraint Methods (SCM) are essential tools for the construction of a lower bound for the inf-sup stability constants which are required in {\it a posteriori} error analysis of reduced basis approximations. They utilize a Linear Program (LP) relaxation scheme incorporating continuity and stability constraints. The natural-norm approach {\em linearizes} inf-sup constant as a function of the parameter. The Natural-Norm Successive Constraint Method (NNSCM) combines these two aspects. It uses a greedy algorithm to select SCM control points which adaptively construct an optimal decomposition of the parameter domain, and then apply the SCM on each domain. Unfortunately, the NNSCM produces no guarantee for the quality of the lower bound. The new cNNSCM provides an upper bound in addition to the lower bound and let the user control the gap, thus the quality of the lower bound. The efficacy and accuracy of the new method is validated by numerical experiments.

Bo Dong, Jiahua Jiang, Yanlai Chen

We develop and analyze the first hybridizable discontinuous Galerkin (HDG) method for solving fifth-order Korteweg-de Vries (KdV) type equations. We show that the semi-discrete scheme is stable with proper choices of the stabilization functions in the numerical traces. For the linearized fifth-order equations, we prove that the approximations to the exact solution and its four spatial derivatives as well as its time derivative all have optimal convergence rates. The numerical experiments, demonstrating optimal convergence rates for both the linear and nonlinear equations, validate our theoretical findings.

Jiahua Jiang, Yanlai Chen, Akil Narayan

The Reduced Basis Method (RBM) is a popular certified model reduction approach for solving parametrized partial differential equations. One critical stage of the \textit{offline} portion of the algorithm is a greedy algorithm, requiring maximization of an error estimate over parameter space. In practice this maximization is usually performed by replacing the parameter domain continuum with a discrete "training" set. When the dimension of parameter space is large, it is necessary to significantly increase the size of this training set in order to effectively search parameter space. Large training sets diminish the attractiveness of RBM algorithms since this proportionally increases the cost of the offline {phase}. In this work we propose novel strategies for offline RBM algorithms that mitigate the computational difficulty of maximizing error estimates over a training set. The main idea is to identify a subset of the training set, a "surrogate parameter domain" (SPD), on which to perform greedy algorithms. The SPD's we construct are much smaller in size than the full training set, yet our examples suggest that they are accurate enough to represent the solution manifold of interest at the current offline RBM iteration. We propose two algorithms to construct the SPD: Our first algorithm, the Successive Maximization Method (SMM) method, is inspired by inverse transform sampling for non-standard univariate probability distributions. The second constructs an SPD by identifying pivots in the Cholesky Decomposition of an approximate error correlation matrix. We demonstrate the algorithm through numerical experiments, showing that the algorithm is capable of accelerating offline RBM procedures without degrading accuracy, assuming that the solution manifold has low Kolmogorov width.

M. Shan Tariq, Yanlai Chen, Bo Dong

We develop new conservative discontinuous Galerkin (DG) methods for nonlinear wave problems, focusing on the generalized Korteweg-de Vries (gKdV) equation and the coupled Hirota-Satsuma KdV (HS-KdV) system. The proposed methods preserve mass through the single-valued structure of numerical traces, while energy and Hamiltonian conservation are enforced by implicitly determining penalty parameters in the numerical traces through auxiliary conservation constraints. In our previous work [11], we developed a conservative DG method for the gKdV equation; however, that formulation involves the time derivative of the jump of the approximate solution, which complicates extensions beyond second-order temporal accuracy. Our new formulation overcomes this limitation by introducing a redesigned trace configuration that eliminates the derivative-of-jump term. This novel enhancement seamlessly paves the way for higher-order time discretizations and requires solving fewer nonlinear systems per time step than the previous approach. For the coupled HS-KdV system, we present the first conservative DG method that preserves all three invariants of the exact solution. Numerical results demonstrate the accuracy and expected convergence behavior of the proposed methods, as well as long-time stability and strong conservation properties for both the gKdV equation and HS-KdV system.

Lijie Ji, Sabrina Rashid, Yanlai Chen et al.

Traditional projection-based reduced-order modeling approximates the full-order model by projecting it onto a linear subspace. With a fast-decaying Kolmogorov $n$-width of the solution manifold, the resulting reduced-order model (ROM) can be an efficient and accurate emulator. However, for parametric partial differential equations with slowly decaying Kolmogorov $n$-width, the dimension of the linear subspace required for a reasonable accuracy becomes very large, which undermines computational efficiency. To address this limitation, quadratic manifold methods have recently been proposed. These data-driven methods first identify a quadratic mapping by minimizing the linear projection error over a large set of snapshots, often with the aid of regularization techniques to solve the associated minimization problem, and then use this mapping to construct ROMs. In this paper, we propose and test a novel enhancement to this quadratic manifold approach by introducing a first-of-its-kind double-greedy algorithm on the regularization parameters coupled with a standard greedy algorithm on the physical parameter. Our approach balances the trade-off between the accuracy of the quadratic mapping and the stability of the resulting nonlinear ROM, leading to a highly efficient and data-sparse algorithm. Numerical experiments conducted on equations such as linear transport, acoustic wave, advection-diffusion, and Burgers' demonstrate the accuracy, efficiency, and stability of the proposed algorithm.

Yanlai Chen, Yajie Ji, Akil Narayan et al.

We introduce the Transformed Generative Pre-Trained Physics-Informed Neural Networks (TGPT-PINN) for accomplishing nonlinear model order reduction (MOR) of transport-dominated partial differential equations in an MOR-integrating PINNs framework. Building on the recent development of the GPT-PINN that is a network-of-networks design achieving snapshot-based model reduction, we design and test a novel paradigm for nonlinear model reduction that can effectively tackle problems with parameter-dependent discontinuities. Through incorporation of a shock-capturing loss function component as well as a parameter-dependent transform layer, the TGPT-PINN overcomes the limitations of linear model reduction in the transport-dominated regime. We demonstrate this new capability for nonlinear model reduction in the PINNs framework by several nontrivial parametric partial differential equations.

Yanlai Chen, Yajie Ji, Zhenli Xu

The Lifshitz-Petrich (LP) model is a classical model for describing complex spatial patterns such as quasicrystals and multiphase structures. Solving and classifying the solutions of the LP model is challenging due to the presence of high-order gradient terms and the long-range orientational order characteristic of the quasicrystals. To address these challenges, we propose a Derivative-informed Graph Convolutional Autoencoder (DiGCA) to classify the multi-component multi-state solutions of the LP model. The classifier consists of two stages. In the offline stage, the DiGCA phase classifier innovatively incorporates both solutions and their derivatives for training a graph convolutional autoencoder which effectively captures intricate spatial dependencies while significantly reducing the dimensionality of the solution space. In the online phase, the framework employs a neural network classifier to efficiently categorize encoded solutions into distinct phase diagrams. The numerical results demonstrate that the DiGCA phase classifier accurately solves the LP model, classifies its solutions, and rapidly generates detailed phase diagrams in a robust manner, offering significant improvements in both efficiency and accuracy over traditional methods.

Haolan Zheng, Yanlai Chen, Jiequn Han et al.

We propose a novel data-lean operator learning algorithm, the Reduced Basis Neural Operator (ReBaNO), to solve a group of PDEs with multiple distinct inputs. Inspired by the Reduced Basis Method and the recently introduced Generative Pre-Trained Physics-Informed Neural Networks, ReBaNO relies on a mathematically rigorous greedy algorithm to build its network structure offline adaptively from the ground up. Knowledge distillation via task-specific activation function allows ReBaNO to have a compact architecture requiring minimal computational cost online while embedding physics. In comparison to state-of-the-art operator learning algorithms such as PCA-Net, DeepONet, FNO, and CNO, numerical results demonstrate that ReBaNO significantly outperforms them in terms of eliminating/shrinking the generalization gap for both in- and out-of-distribution tests and being the only operator learning algorithm achieving strict discretization invariance.

Yajie Ji, Yanlai Chen, Shawn Koohy

We propose S$^2$GPT-PINN, a sparse and small model for solving parametric partial differential equations (PDEs). Similar to Small Language Models (SLMs), S$^2$GPT-PINN is tailored to domain-specific (families of) PDEs and characterized by its compact architecture and minimal computational power. Leveraging a small amount of extremely high quality data via a mathematically rigorous greedy algorithm that is enabled by the large full-order models, S$^2$GPT-PINN relies on orders of magnitude less parameters than PINNs to achieve extremely high efficiency via two levels of customizations. The first is knowledge distillation via task-specific activation functions that are transferred from Pre-Trained PINNs. The second is a judicious down-sampling when calculating the physics-informed loss of the network compressing the number of data sites by orders of magnitude to the size of the small model.

Shijin Hou, Yanlai Chen, Yinhua Xia

Repeatedly solving the parameterized optimal mass transport (pOMT) problem is a frequent task in applications such as image registration and adaptive grid generation. It is thus critical to develop a highly efficient reduced solver that is equally accurate as the full order model. In this paper, we propose such a machine learning-like method for pOMT by adapting a new reduced basis (RB) technique specifically designed for nonlinear equations, the reduced residual reduced over-collocation (R2-ROC) approach, to the parameterized Monge-Amp$\grave{\rm e}$re equation. It builds on top of a narrow-stencil finite different method (FDM), a so-called truth solver, which we propose in this paper for the Monge-Amp$\grave{\rm e}$re equation with a transport boundary. Together with the R2-ROC approach, it allows us to handle the strong and unique nonlinearity pertaining to the Monge-Amp$\grave{\rm e}$re equation achieving online efficiency without resorting to any direct approximation of the nonlinearity. Several challenging numerical tests demonstrate the accuracy and high efficiency of our method for solving the Monge-Amp$\grave{\rm e}$re equation with various parametric boundary conditions.

Huy Dinh, Harbir Antil, Yanlai Chen et al.

We propose a novel numerical algorithm utilizing model reduction for computing solutions to stationary partial differential equations involving the spectral fractional Laplacian. Our approach utilizes a known characterization of the solution in terms of an integral of solutions to classical elliptic problems. We reformulate this integral into an expression whose continuous and discrete formulations are stable; the discrete formulations are stable independent of all discretization parameters. We subsequently apply the reduced basis method to accomplish model order reduction for the integrand. Our choice of quadrature in discretization of the integral is a global Gaussian quadrature rule that we observe is more efficient than previously proposed quadrature rules. Finally, the model reduction approach enables one to compute solutions to multi-query fractional Laplace problems with order of magnitude less cost than a traditional solver.

Yanlai Chen, Jiahua Jiang, Akil Narayan

The Reduced Basis Method (RBM) is a rigorous model reduction approach for solving parametrized partial differential equations. It identifies a low-dimensional subspace for approximation of the parametric solution manifold that is embedded in high-dimensional space. A reduced order model is subsequently constructed in this subspace. RBM relies on residual-based error indicators or {\em a posteriori} error bounds to guide construction of the reduced solution subspace, to serve as a stopping criteria, and to certify the resulting surrogate solutions. Unfortunately, it is well-known that the standard algorithm for residual norm computation suffers from premature stagnation at the level of the square root of machine precision. In this paper, we develop two alternatives to the standard offline phase of reduced basis algorithms. First, we design a robust strategy for computation of residual error indicators that allows RBM algorithms to enrich the solution subspace with accuracy beyond root machine precision. Secondly, we propose a new error indicator based on the Lebesgue function in interpolation theory. This error indicator does not require computation of residual norms, and instead only requires the ability to compute the RBM solution. This residual-free indicator is rigorous in that it bounds the error committed by the RBM approximation, but up to an uncomputable multiplicative constant. Because of this, the residual-free indicator is effective in choosing snapshots during the offline RBM phase, but cannot currently be used to certify error that the approximation commits. However, it circumvents the need for \textit{a posteriori} analysis of numerical methods, and therefore can be effective on problems where such a rigorous estimate is hard to derive.

Harbir Antil, Yanlai Chen, Akil Narayan

Fractional Laplace equations are becoming important tools for mathematical modeling and prediction. Recent years have shown much progress in developing accurate and robust algorithms to numerically solve such problems, yet most solvers for fractional problems are computationally expensive. Practitioners are often interested in choosing the fractional exponent of the mathematical model to match experimental and/or observational data; this requires the computational solution to the fractional equation for several values of the both exponent and other parameters that enter the model, which is a computationally expensive many-query problem. To address this difficulty, we present a model order reduction strategy for fractional Laplace problems utilizing the reduced basis method (RBM). Our RBM algorithm for this fractional partial differential equation (PDE) allows us to accomplish significant acceleration compared to a traditional PDE solver while maintaining accuracy. Our numerical results demonstrate this accuracy and efficiency of our RBM algorithm on fractional Laplace problems in two spatial dimensions.

Jiahua Jiang, Yanlai Chen, Akil Narayan

The non-intrusive generalized Polynomial Chaos (gPC) method is a popular computational approach for solving partial differential equations (PDEs) with random inputs. The main hurdle preventing its efficient direct application for high-dimensional input parameters is that the size of many parametric sampling meshes grows exponentially in the number of inputs (the "curse of dimensionality"). In this paper, we design a weighted version of the reduced basis method (RBM) for use in the non-intrusive gPC framework. We construct an RBM surrogate that can rigorously achieve a user-prescribed error tolerance, and ultimately is used to more efficiently compute a gPC approximation non-intrusively. The algorithm is capable of speeding up traditional non-intrusive gPC methods by orders of magnitude without degrading accuracy, assuming that the solution manifold has low Kolmogorov width. Numerical experiments on our test problems show that the relative efficiency improves as the parametric dimension increases, demonstrating the potential of the method in delaying the curse of dimensionality. Theoretical results as well as numerical evidence justify these findings.

Yanlai Chen

Dimension reduction is often needed in the area of data mining. The goal of these methods is to map the given high-dimensional data into a low-dimensional space preserving certain properties of the initial data. There are two kinds of techniques for this purpose. The first, projective methods, builds an explicit linear projection from the high-dimensional space to the low-dimensional one. On the other hand, the nonlinear methods utilizes nonlinear and implicit mapping between the two spaces. In both cases, the methods considered in literature have usually relied on computationally very intensive matrix factorizations, frequently the Singular Value Decomposition (SVD). The computational burden of SVD quickly renders these dimension reduction methods infeasible thanks to the ever-increasing sizes of the practical datasets. In this paper, we present a new decomposition strategy, Reduced Basis Decomposition (RBD), which is inspired by the Reduced Basis Method (RBM). Given $X$ the high-dimensional data, the method approximates it by $Y \, T (\approx X)$ with $Y$ being the low-dimensional surrogate and $T$ the transformation matrix. $Y$ is obtained through a greedy algorithm thus extremely efficient. In fact, it is significantly faster than SVD with comparable accuracy. $T$ can be computed on the fly. Moreover, unlike many compression algorithms, it easily finds the mapping for an arbitrary ``out-of-sample'' vector and it comes with an ``error indicator'' certifying the accuracy of the compression. Numerical results are shown validating these claims.