Zhiping Mao

Bin Lin, Zhiping Mao, Zhicheng Wang et al.

Physics-informed Neural Networks (PINNs) have been shown as a promising approach for solving both forward and inverse problems of partial differential equations (PDEs). Meanwhile, the neural operator approach, including methods such as Deep Operator Network (DeepONet) and Fourier neural operator (FNO), has been introduced and extensively employed in approximating solution of PDEs. Nevertheless, to solve problems consisting of sharp solutions poses a significant challenge when employing these two approaches. To address this issue, we propose in this work a novel framework termed Operator Learning Enhanced Physics-informed Neural Networks (OL-PINN). Initially, we utilize DeepONet to learn the solution operator for a set of smooth problems relevant to the PDEs characterized by sharp solutions. Subsequently, we integrate the pre-trained DeepONet with PINN to resolve the target sharp solution problem. We showcase the efficacy of OL-PINN by successfully addressing various problems, such as the nonlinear diffusion-reaction equation, the Burgers equation and the incompressible Navier-Stokes equation at high Reynolds number. Compared with the vanilla PINN, the proposed method requires only a small number of residual points to achieve a strong generalization capability. Moreover, it substantially enhances accuracy, while also ensuring a robust training process. Furthermore, OL-PINN inherits the advantage of PINN for solving inverse problems. To this end, we apply the OL-PINN approach for solving problems with only partial boundary conditions, which usually cannot be solved by the classical numerical methods, showing its capacity in solving ill-posed problems and consequently more complex inverse problems.

Nan Wang, Zhiping Mao, Chengming Huang et al.

We consider spectral approximations to the conservative form of the two-sided Riemann-Liouville (R-L) and Caputo fractional differential equations (FDEs) with nonhomogeneous Dirichlet (fractional and classical, respectively) and Neumann (fractional) boundary conditions. In particular, we develop a spectral penalty method (SPM) by using the Jacobi poly-fractonomial approximation for the conservative R-L FDEs while using the polynomial approximation for the conservative Caputo FDEs. We establish the well-posedness of the corresponding weak problems and analyze sufficient conditions for the coercivity of the SPM for different types of fractional boundary value problems. This analysis allows us to estimate the proper values of the penalty parameters at boundary points. We present several numerical examples to verify the theory and demonstrate the high accuracy of SPM, both for stationary and time dependent FDEs. Moreover, we compare the results against a Petrov-Galerkin spectral tau method (PGS-$τ$, an extension of [Z. Mao, G.E. Karniadakis, SIAM J. Numer. Anal., 2018]) and demonstrate the superior accuracy of SPM for all cases considered.

Xianjuan Li, Zhiping Mao, Fangying Song et al.

We develop a fast solver for the spectral element method (SEM) applied to the two-sided fractional diffusion equation on uniform, geometric and graded meshes. By approximating the singular kernel with a degenerate kernel, we construct a hierarchical matrix (H-matrix) to represent the stiffness matrix of the SEM and provide error estimates verified numerically. We can solve efficiently the H-matrix approximation problem using a hierarchical LU decomposition method, which reduces the computational cost to $O(R^2 N_d \log^2N) +O(R^3 N_d \log N)$, where $R$ it is the rank of submatrices of the H-matrix approximation, $N_d$ is the total number of degrees of freedom and $N$ is the number of elements. However, we lose the high accuracy of the SEM. Thus, we solve the corresponding preconditioned system by using the H-matrix approximation problem as a preconditioner, recovering the high order accuracy of the SEM. The condition number of the preconditioned system is independent of the polynomial degree $P$ and grows with the number of elements, but at modest values of the rank $R$ is below order 10 in our experiments, which represents a reduction of more than 11 orders of magnitude from the unpreconditioned system; this reduction is higher in the two-sided fractional derivative compared to one-sided fractional derivative. The corresponding cost is $O(R^2 N_d \log^2 N)+O(R^3 N_d \log N)+O(N_d^2)$. Moreover, by using a structured mesh (uniform or geometric mesh), we can further reduce the computational cost to $O(R^2 N_d\log^2 N) +O(R^3 N_d \log N)+ O(P^2 N\log N)$ for the preconditioned system. We present several numerical tests to illustrate the proposed algorithm using $h$ and $p$ refinements.

Lizhen Chen, Zhiping Mao, Huiyuan Li

An efficient Jacobi-Galerkin spectral method for calculating eigenvalues of Riesz fractional partial differential equations with homogeneous Dirichlet boundary values is proposed in this paper. In order to retain the symmetry and positive definiteness of the discrete linear system, we introduce some properly defined Sobolev spaces and approximate the eigenvalue problem in a standard Galerkin weak formulation instead of the Petrov-Galerkin one as in literature. Poincaré and inverse inequalities are proved for the proposed Galerkin formulation which finally help us establishing a sharp estimate on the algebraic system's condition number. Rigorous error estimates of the eigenvalues and eigenvectors are then readily obtained by using Babuška and Osborn's approximation theory on self-adjoint and positive-definite eigenvalue problems. Numerical results are presented to demonstrate the accuracy and efficiency, and to validate the asymptotically exponential oder of convergence. Moreover, the Weyl-type asymptotic law $ λ_n=\mathcal{O}(n^{2α})$ for the $n$-th eigenvalue $λ_n$ of the Riesz fractional differential operator of order $2α$, and the condition number $N^{4α}$ of its algebraic system with respect to the polynomial degree $N$ are observed.

Xiangnan Yu, Hao Xu, Zhiping Mao et al.

In complex physical systems, conventional differential equations often fall short in capturing non-local and memory effects, as they are limited to local dynamics and integer-order interactions. This study introduces a stepwise data-driven framework for discovering fractional differential equations (FDEs) directly from data. FDEs, known for their capacity to model non-local dynamics with fewer parameters than integer-order derivatives, can represent complex systems with long-range interactions. Our framework applies deep neural networks as surrogate models for denoising and reconstructing sparse and noisy observations while using Gaussian-Jacobi quadrature to handle the challenges posed by singularities in fractional derivatives. To optimize both the sparse coefficients and fractional order, we employ an alternating optimization approach that combines sparse regression with global optimization techniques. We validate the framework across various datasets, including synthetic anomalous diffusion data, experimental data on the creep behavior of frozen soils, and single-particle trajectories modeled by Lévy motion. Results demonstrate the framework's robustness in identifying the structure of FDEs across diverse noise levels and its capacity to capture integer-order dynamics, offering a flexible approach for modeling memory effects in complex systems.

Zhangyong Liang, Zhiping Mao, Xiaofei Zhao

In this paper, we propose a neural multiscale decomposition method (NeuralMD) for solving the nonlinear Klein-Gordon equation (NKGE) with a dimensionless parameter $\varepsilon\in(0,1]$ from the relativistic regime to the nonrelativistic limit regime. The solution of the NKGE propagates waves with wavelength at $O(1)$ and $O(\varepsilon^2)$ in space and time, respectively, which brings the oscillation in time. Existing collocation-based methods for solving this equation lead to spectral bias and propagation failure. To mitigate the spectral bias induced by high-frequency time oscillation, we employ a multiscale time integrator (MTI) to absorb the time oscillation into the phase. This decomposes the NKGE into a nonlinear Schrödinger equation with wave operator (NLSW) with well-prepared initial data and a remainder equation with small initial data. As $\varepsilon \to 0$, the NKGE converges to the NLSW at rate $O(\varepsilon^{2})$, and the contribution of the remainder equation becomes negligible. Furthermore, to alleviate propagation failure caused by medium-frequency time oscillation, we propose a gated gradient correlation correction strategy to enforce temporal coherence in collocation-based methods. As a result, the approximation of the remainder term is no longer affected by propagation failure. Comparative experiments with existing collocation-based methods demonstrate the superior performance of our method for solving the NKGE with various regularities of initial data over the whole regime.

Ameya D. Jagtap, Zhiping Mao, Nikolaus Adams et al.

Accurate solutions to inverse supersonic compressible flow problems are often required for designing specialized aerospace vehicles. In particular, we consider the problem where we have data available for density gradients from Schlieren photography as well as data at the inflow and part of wall boundaries. These inverse problems are notoriously difficult and traditional methods may not be adequate to solve such ill-posed inverse problems. To this end, we employ the physics-informed neural networks (PINNs) and its extended version, extended PINNs (XPINNs), where domain decomposition allows deploying locally powerful neural networks in each subdomain, which can provide additional expressivity in subdomains, where a complex solution is expected. Apart from the governing compressible Euler equations, we also enforce the entropy conditions in order to obtain viscosity solutions. Moreover, we enforce positivity conditions on density and pressure. We consider inverse problems involving two-dimensional expansion waves, two-dimensional oblique and bow shock waves. We compare solutions obtained by PINNs and XPINNs and invoke some theoretical results that can be used to decide on the generalization errors of the two methods.

Xuhui Meng, Liu Yang, Zhiping Mao et al.

We develop a new Bayesian framework based on deep neural networks to be able to extrapolate in space-time using historical data and to quantify uncertainties arising from both noisy and gappy data in physical problems. Specifically, the proposed approach has two stages: (1) prior learning and (2) posterior estimation. At the first stage, we employ the physics-informed Generative Adversarial Networks (PI-GAN) to learn a functional prior either from a prescribed function distribution, e.g., Gaussian process, or from historical data and physics. At the second stage, we employ the Hamiltonian Monte Carlo (HMC) method to estimate the posterior in the latent space of PI-GANs. In addition, we use two different approaches to encode the physics: (1) automatic differentiation, used in the physics-informed neural networks (PINNs) for scenarios with explicitly known partial differential equations (PDEs), and (2) operator regression using the deep operator network (DeepONet) for PDE-agnostic scenarios. We then test the proposed method for (1) meta-learning for one-dimensional regression, and forward/inverse PDE problems (combined with PINNs); (2) PDE-agnostic physical problems (combined with DeepONet), e.g., fractional diffusion as well as saturated stochastic (100-dimensional) flows in heterogeneous porous media; and (3) spatial-temporal regression problems, i.e., inference of a marine riser displacement field. The results demonstrate that the proposed approach can provide accurate predictions as well as uncertainty quantification given very limited scattered and noisy data, since historical data could be available to provide informative priors. In summary, the proposed method is capable of learning flexible functional priors, and can be extended to big data problems using stochastic HMC or normalizing flows since the latent space is generally characterized as low dimensional.

Shengze Cai, Zhiping Mao, Zhicheng Wang et al.

Despite the significant progress over the last 50 years in simulating flow problems using numerical discretization of the Navier-Stokes equations (NSE), we still cannot incorporate seamlessly noisy data into existing algorithms, mesh-generation is complex, and we cannot tackle high-dimensional problems governed by parametrized NSE. Moreover, solving inverse flow problems is often prohibitively expensive and requires complex and expensive formulations and new computer codes. Here, we review flow physics-informed learning, integrating seamlessly data and mathematical models, and implementing them using physics-informed neural networks (PINNs). We demonstrate the effectiveness of PINNs for inverse problems related to three-dimensional wake flows, supersonic flows, and biomedical flows.

Lu Lu, Xuhui Meng, Zhiping Mao et al.

Deep learning has achieved remarkable success in diverse applications; however, its use in solving partial differential equations (PDEs) has emerged only recently. Here, we present an overview of physics-informed neural networks (PINNs), which embed a PDE into the loss of the neural network using automatic differentiation. The PINN algorithm is simple, and it can be applied to different types of PDEs, including integro-differential equations, fractional PDEs, and stochastic PDEs. Moreover, from the implementation point of view, PINNs solve inverse problems as easily as forward problems. We propose a new residual-based adaptive refinement (RAR) method to improve the training efficiency of PINNs. For pedagogical reasons, we compare the PINN algorithm to a standard finite element method. We also present a Python library for PINNs, DeepXDE, which is designed to serve both as an education tool to be used in the classroom as well as a research tool for solving problems in computational science and engineering. Specifically, DeepXDE can solve forward problems given initial and boundary conditions, as well as inverse problems given some extra measurements. DeepXDE supports complex-geometry domains based on the technique of constructive solid geometry, and enables the user code to be compact, resembling closely the mathematical formulation. We introduce the usage of DeepXDE and its customizability, and we also demonstrate the capability of PINNs and the user-friendliness of DeepXDE for five different examples. More broadly, DeepXDE contributes to the more rapid development of the emerging Scientific Machine Learning field.