Jian-Guo Liu

Jian-Guo Liu, Li Wang, Zhennan Zhou

We propose a semi-discrete scheme for 2D Keller-Segel equations based on a symmetrization reformation, which is equivalent to the convex splitting method and is free of any nonlinear solver. We show that, this new scheme is unconditionally stable as long as the initial condition does not exceed certain threshold, and it asymptotically preserves the quasi-static limit in the transient regime. Furthermore, we prove that the fully discrete scheme is conservative and positivity preserving, which makes it ideal for simulations. The analogical schemes for the radial symmetric cases and the subcritical degenerate cases are also presented and analyzed. With extensive numerical tests, we verify the claimed properties of the methods and demonstrate their superiority in various challenging applications.

Philippe G. LeFloch, Jian-Guo Liu

Solutions to conservation laws satisfy the monotonicity property: the number of local extrema is a non-increasing function of time, and local maximum/minimum values decrease/increase monotonically in time. This paper investigates this property from a numerical standpoint. We introduce a class of fully discrete in space and time, high order accurate, difference schemes, called generalized monotone schemes. Convergence toward the entropy solution is proven via a new technique of proof, assuming that the initial data has a finite number of extremum values only, and the flux-function is strictly convex. We define discrete paths of extrema by tracking local extremum values in the approximate solution. In the course of the analysis we establish the pointwise convergence of the trace of the solution along a path of extremum. As a corollary, we obtain a proof of convergence for a MUSCL-type scheme being second order accurate away from sonic points and extrema.

Jian-Guo Liu, Min Tang, Li Wang et al.

In this paper, we study the tumor growth equation along with various models for the nutrient component, including the \emph{in vitro} model and the \emph{in vivo} model. At the cell density level, the spatial availability of the tumor density $n$ is governed by the Darcy law via the pressure $p(n)=n^γ$. For finite $γ$, we prove some a priori estimates of the tumor growth model, such as boundedness of the nutrient density, and non-negativity and growth estimate of the tumor density. As $γ\rightarrow \infty$, the cell density models formally converge to Hele-Shaw flow models, which determine the free boundary dynamics of the tumor tissue in the incompressible limit. We derive several analytical solutions to the Hele-Shaw flow models, which serve as benchmark solutions to the geometric motion of tumor front propagation. Finally, we apply a conservative and positivity preserving numerical scheme to the cell density models, with numerical results verifying the link between cell density models and the free boundary dynamical models.

Jian-Guo Liu, Zheng Ma, Zhennan Zhou

In this paper, we investigate numerical approximations of the scalar conservation law with the Caputo derivative, which introduces the memory effect. We construct the first order and the second order explicit upwind schemes for such equations, which are shown to be conditionally $\ell^1$ contracting and TVD. However, the Caputo derivative leads to the modified CFL-type stability condition, $ (Δt)^α = O(Δx)$, where $α\in (0,1]$ is the fractional exponent in the derivative. When $α$ small, such strong constraint makes the numerical implementation extremely impractical. We have then proposed the implicit upwind scheme to overcome this issue, which is proved to be unconditionally $\ell^1$ contracting and TVD. Various numerical tests are presented to validate the properties of the methods and provide more numerical evidence in interpreting the memory effect in conservation laws.

Lei Li, Jian-Guo Liu

We revisit the traditional upwind schemes for linear conservation laws in the viewpoint of jump processes, allowing studying upwind schemes using probabilistic tools. In particular, for Fokker-Planck equations on $\mathbb{R}$, in the case of weak confinement, we show that the solution of upwind scheme converges to a stationary solution. In the case of strong confinement, using a discrete Poincaré inequality, we prove that the $O(h)$ numeric error under $\ell^1$ norm is uniform in time, and establish the uniform exponential convergence to the steady states. Compared with the traditional results of exponential convergence of upwind schemes, our result is in the whole space without boundary. We also establish similar results on torus for which the stationary solution of the scheme does not have detailed balance. This work shows an interesting connection between standard numerical methods and time continuous Markov chains, and could motivate better understanding of numerical analysis for conservation laws.

Lei Li, Jian-Guo Liu, Yuliang Wang

We consider the geometric ergodicity of the Stochastic Gradient Langevin Dynamics (SGLD) algorithm under nonconvexity settings. Via the technique of reflection coupling, we prove the Wasserstein contraction of SGLD when the target distribution is log-concave only outside some compact set. The time discretization and the minibatch in SGLD introduce several difficulties when applying the reflection coupling, which are addressed by a series of careful estimates of conditional expectations. As a direct corollary, the SGLD with constant step size has an invariant distribution and we are able to obtain its geometric ergodicity in terms of $W_1$ distance. The generalization to non-gradient drifts is also included.

Lei Li, Jian-Guo Liu, Zibu Liu et al.

There is a growing interest in investigating numerical approximations of the water wave equation in recent years, whereas the lack of rigorous analysis of its time discretization inhibits the design of more efficient algorithms. In this work, we focus on a nonlocal hyperbolic model, which essentially inherits the features of the water wave equation, and is simplified from the latter. For the constant coefficient case, we carry out systematical stability studies of the fully discrete approximation of such systems with the Fourier spectral approximation in space and general Runge-Kutta method in time. In particular, we discover the optimal time step constraints, in the form of a modified CFL condition, when certain explicit Runge-Kutta method are applied. Besides, the convergence of the semi-discrete approximation of variable coefficient case is shown, which naturally connects to the water wave equation. Extensive numerical tests have been performed to verify the stability conditions and simulations of the simplified hyperbolic model in the high frequency regime and the water wave equation are also provided.

Yuqi Wang, Aarzu Gupta, David Carpenter et al.

TodevelopanovelUncertaintyQuantification (UQ) framework to estimate the uncertainty of patient survival models in the absence of ground truth, we developed and evaluated our approach based on a dataset of 1383 patients treated with stereotactic radiosurgery (SRS) for brain metastases between January 2015 and December 2020. Our motivating hypothesis is that a time-to-event prediction of a test patient on inference is more certain given a higher feature-space-similarity to patients in the training set. Therefore, the uncertainty for a particular patient-of-interest is represented by the concordance index between a patient similarity rank and a prediction similarity rank. Model uncertainty was defined as the increased percentage of the max uncertainty-constrained-AUC compared to the model AUC. We evaluated our method on multiple clinically-relevant endpoints, including time to intracranial progression (ICP), progression-free survival (PFS) after SRS, overall survival (OS), and time to ICP and/or death (ICPD), on a variety of both statistical and non-statistical models, including CoxPH, conditional survival forest (CSF), and neural multi-task linear regression (NMTLR). Our results show that all models had the lowest uncertainty on ICP (2.21%) and the highest uncertainty (17.28%) on ICPD. OS models demonstrated high variation in uncertainty performance, where NMTLR had the lowest uncertainty(1.96%)and CSF had the highest uncertainty (14.29%). In conclusion, our method can estimate the uncertainty of individual patient survival modeling results. As expected, our data empirically demonstrate that as model uncertainty measured via our technique increases, the similarity between a feature-space and its predicted outcome decreases.

Yuan Gao, Jian-Guo Liu, Nan Wu

We study the Langevin dynamics of a physical system with manifold structure $\mathcal{M}\subset\mathbb{R}^p$ based on collected sample points $\{\mathsf{x}_i\}_{i=1}^n \subset \mathcal{M}$ that probe the unknown manifold $\mathcal{M}$. Through the diffusion map, we first learn the reaction coordinates $\{\mathsf{y}_i\}_{i=1}^n\subset \mathcal{N}$ corresponding to $\{\mathsf{x}_i\}_{i=1}^n$, where $\mathcal{N}$ is a manifold diffeomorphic to $\mathcal{M}$ and isometrically embedded in $\mathbb{R}^\ell$ with $\ell \ll p$. The induced Langevin dynamics on $\mathcal{N}$ in terms of the reaction coordinates captures the slow time scale dynamics such as conformational changes in biochemical reactions. To construct an efficient and stable approximation for the Langevin dynamics on $\mathcal{N}$, we leverage the corresponding Fokker-Planck equation on the manifold $\mathcal{N}$ in terms of the reaction coordinates $\mathsf{y}$. We propose an implementable, unconditionally stable, data-driven finite volume scheme for this Fokker-Planck equation, which automatically incorporates the manifold structure of $\mathcal{N}$. Furthermore, we provide a weighted $L^2$ convergence analysis of the finite volume scheme to the Fokker-Planck equation on $\mathcal{N}$. The proposed finite volume scheme leads to a Markov chain on $\{\mathsf{y}_i\}_{i=1}^n$ with an approximated transition probability and jump rate between the nearest neighbor points. After an unconditionally stable explicit time discretization, the data-driven finite volume scheme gives an approximated Markov process for the Langevin dynamics on $\mathcal{N}$ and the approximated Markov process enjoys detailed balance, ergodicity, and other good properties.

Lei Li, Yingzhou Li, Jian-Guo Liu et al.

We propose in this work RBM-SVGD, a stochastic version of Stein Variational Gradient Descent (SVGD) method for efficiently sampling from a given probability measure and thus useful for Bayesian inference. The method is to apply the Random Batch Method (RBM) for interacting particle systems proposed by Jin et al to the interacting particle systems in SVGD. While keeping the behaviors of SVGD, it reduces the computational cost, especially when the interacting kernel has long range. Numerical examples verify the efficiency of this new version of SVGD.

Yuanyuan Feng, Tingran Gao, Lei Li et al.

Diffusion approximation provides weak approximation for stochastic gradient descent algorithms in a finite time horizon. In this paper, we introduce new tools motivated by the backward error analysis of numerical stochastic differential equations into the theoretical framework of diffusion approximation, extending the validity of the weak approximation from finite to infinite time horizon. The new techniques developed in this paper enable us to characterize the asymptotic behavior of constant-step-size SGD algorithms for strongly convex objective functions, a goal previously unreachable within the diffusion approximation framework. Our analysis builds upon a truncated formal power expansion of the solution of a stochastic modified equation arising from diffusion approximation, where the main technical ingredient is a uniform-in-time weak error bound controlling the long-term behavior of the expansion coefficient functions near the global minimum. We expect these new techniques to greatly expand the range of applicability of diffusion approximation to cover wider and deeper aspects of stochastic optimization algorithms in data science.

Jian-Guo Liu, Min Tang, Li Wang et al.

We consider a class of tumor growth models under the combined effects of density-dependent pressure and cell multiplication, with a free boundary model as its singular limit when the pressure-density relationship becomes highly nonlinear. In particular, the constitutive law connecting pressure $p$ and density $ρ$ is $p(ρ)=\frac{m}{m-1} ρ^{m-1}$, and when $m \gg 1$, the cell density $ρ$ may evolve its support due to a pressure-driven geometric motion with sharp interface along the boundary of its support. The nonlinearity and degeneracy in the diffusion bring great challenges in numerical simulations, let alone the capturing of the singular free boundary limit. Prior to the present paper, there is lack of standard mechanism to numerically capture the front propagation speed as $m\gg 1$. In this paper, we develope a numerical scheme based on a novel prediction-correction reformulation that can accurately approximate the front propagation even when the nonlinearity is extremely strong. We show that the semi-discrete scheme naturally connects to the free boundary limit equation as $m \rightarrow \infty$, and with proper spacial discretization, the fully discrete scheme has improved stability, preserves positivity, and implements without nonlinear solvers. Finally, extensive numerical examples in both one and two dimensions are provided to verify the claimed properties and showcase good performance in various applications.

Wenqing Hu, Chris Junchi Li, Lei Li et al.

We study the Stochastic Gradient Descent (SGD) method in nonconvex optimization problems from the point of view of approximating diffusion processes. We prove rigorously that the diffusion process can approximate the SGD algorithm weakly using the weak form of master equation for probability evolution. In the small step size regime and the presence of omnidirectional noise, our weak approximating diffusion process suggests the following dynamics for the SGD iteration starting from a local minimizer (resp.~saddle point): it escapes in a number of iterations exponentially (resp.~almost linearly) dependent on the inverse stepsize. The results are obtained using the theory for random perturbations of dynamical systems (theory of large deviations for local minimizers and theory of exiting for unstable stationary points). In addition, we discuss the effects of batch size for the deep neural networks, and we find that small batch size is helpful for SGD algorithms to escape unstable stationary points and sharp minimizers. Our theory indicates that one should increase the batch size at later stage for the SGD to be trapped in flat minimizers for better generalization.

Qiang Guo, Wen-Jun Song, Jian-Guo Liu

A key challenge of the collaborative filtering (CF) information filtering is how to obtain the reliable and accurate results with the help of peers' recommendation. Since the similarities from small-degree users to large-degree users would be larger than the ones opposite direction, the large-degree users' selections are recommended extensively by the traditional second-order CF algorithms. By considering the users' similarity direction and the second-order correlations to depress the influence of mainstream preferences, we present the directed second-order CF (HDCF) algorithm specifically to address the challenge of accuracy and diversity of the CF algorithm. The numerical results for two benchmark data sets, MovieLens and Netflix, show that the accuracy of the new algorithm outperforms the state-of-the-art CF algorithms. Comparing with the CF algorithm based on random-walks proposed in the Ref.7, the average ranking score could reach 0.0767 and 0.0402, which is enhanced by 27.3\% and 19.1\% for MovieLens and Netflix respectively. In addition, the diversity, precision and recall are also enhanced greatly. Without relying on any context-specific information, tuning the similarity direction of CF algorithms could obtain accurate and diverse recommendations. This work suggests that the user similarity direction is an important factor to improve the personalized recommendation performance.

Jian-Guo Liu, Kerui Shi, Qiang Guo

Random walks have been successfully used to measure user or object similarities in collaborative filtering (CF) recommender systems, which is of high accuracy but low diversity. A key challenge of CF system is that the reliably accurate results are obtained with the help of peers' recommendation, but the most useful individual recommendations are hard to be found among diverse niche objects. In this paper we investigate the direction effect of the random walk on user similarity measurements and find that the user similarity, calculated by directed random walks, is reverse to the initial node's degree. Since the ratio of small-degree users to large-degree users is very large in real data sets, the large-degree users' selections are recommended extensively by traditional CF algorithms. By tuning the user similarity direction from neighbors to the target user, we introduce a new algorithm specifically to address the challenge of diversity of CF and show how it can be used to solve the accuracy-diversity dilemma. Without relying on any context-specific information, we are able to obtain accurate and diverse recommendations, which outperforms the state-of-the-art CF methods. This work suggests that the random walk direction is an important factor to improve the personalized recommendation performance.

Jian-Guo Liu, Luc Mieussens

We present a mathematical analysis of the asymptotic preserving scheme proposed in [M. Lemou and L. Mieussens, SIAM J. Sci. Comput., 31, pp. 334-368, 2008] for linear transport equations in kinetic and diffusive regimes. We prove that the scheme is uniformly stable and accurate with respect to the mean free path of the particles. This property is satisfied under an explicitly given CFL condition. This condition tends to a parabolic CFL condition for small mean free paths, and is close to a convection CFL condition for large mean free paths. Ou r analysis is based on very simple energy estimates.

Jian-Guo Liu, Jie Liu, Robert L. Pego

We show that in bounded domains with no-slip boundary conditions, the Navier-Stokes pressure can be determined in a such way that it is strictly dominated by viscosity. As a consequence, in a general domain we can treat the Navier-Stokes equations as a perturbed vector diffusion equation, instead of as a perturbed Stokes system. To illustrate the advantages of this view, we provide a simple proof of the unconditional stability of a difference scheme that is implicit only in viscosity and explicit in both pressure and convection terms, requiring no solution of stationary Stokes systems or inf-sup conditions.