Jinqiao Duan

Aijun Du, Jinqiao Duan

Complex systems display variability over a broad range of spatial and temporal scales. Some scales are unresolved due to computational limitations. The impact of these unresolved scales on the resolved scales needs to be parameterized or taken into account. One stochastic parameterization scheme is devised to take the effects of unresolved scales into account, in the context of solving a nonlinear partial differential equation with memory (a time-integral term), via large eddy simulations. The obtained large eddy simulation model is a stochastic partial differential equation. Numerical experiments are performed to compare the solutions of the original system and of the stochastic large eddy simulation model.

Ting Gao, Jinqiao Duan, Xiaofan Li et al.

The mean first exit time and escape probability are utilized to quantify dynamical behaviors of stochastic differential equations with non-Gaussian alpha-stable type Levy motions. Both deterministic quantities are characterized by differential-integral equations(i.e.,differential equations with non local terms) but with different exterior conditions. The non-Gaussianity of noises manifests as nonlocality at the level of mean exit time and escape probability. An objective of this paper is to make mean exit time and escape probability as efficient computational tools, to the applied probability community, for quantifying stochastic dynamics. An accurate numerical scheme is developed and validated for computing the mean exit time and escape probability. Asymptotic solution for the mean exit time is given when the pure jump measure in the Levy motion is small. From both the analytical and numerical results, it is observed that the mean exit time depends strongly on the domain size and the value of alpha in the alpha-stable Levy jump measure. The mean exit time can measure which of the two competing factors in alpha-stable Levy motion, i.e. the jump frequency or the jump size, is dominant in helping a process exit a bounded domain. The escape probability is shown to vary with the underlying vector field(i.e.,drift). The mean exit time and escape probability could become discontinuous at the boundary of the domain, when the process is subject to certain deterministic potential and the value of alpha is in (0,1).

Xiao Wang, Wenpeng Shang, Xiaofan Li et al.

Non-Gaussian Lévy noises are present in many models for understanding underlining principles of physics, finance, biology and more. In this work, we consider the Fokker-Planck equation(FPE) due to one-dimensional asymmetric Lévy motion, which is a nonlocal partial differential equation. We present an accurate numerical quadrature for the singular integrals in the nonlocal FPE and develop a fast summation method to reduce the order of the complexity from $O(J^2)$ to $O(J\log J)$ in one time-step, where $J$ is the number of unknowns. We also provide conditions under which the numerical schemes satisfy maximum principle. Our numerical method is validated by comparing with exact solutions for special cases. We also discuss the properties of the probability density functions and the effects of various factors on the solutions, including the stability index, the skewness parameter, the drift term, the Gaussian and non-Gaussian noises and the domain size.

Jinqiao Duan

Model uncertainties and simulation uncertainties occur in mathematical modeling of multiscale complex systems, since some mechanisms or scales are not represented (i.e., "unresolved") due to lack in our understanding of these mechanisms or limitations in computational power. The impact of these unresolved scales on the resolved scales needs to be parameterized or taken into account. A stochastic scheme is devised to take the effects of unresolved scales into account, in the context of solving nonlinear partial differential equations. An example is presented to demonstrate this strategy.

Jin Guo, Ting Gao, Yufu Lan et al.

Stochastic Gumbel graph networks are proposed to learn high-dimensional time series, where the observed dimensions are often spatially correlated. To that end, the observed randomness and spatial-correlations are captured by learning the drift and diffusion terms of the stochastic differential equation with a Gumble matrix embedding, respectively. In particular, this novel framework enables us to investigate the implicit regularization effect of the noise terms in S-GGNs. We provide a theoretical guarantee for the proposed S-GGNs by deriving the difference between the two corresponding loss functions in a small neighborhood of weight. Then, we employ Kuramoto's model to generate data for comparing the spectral density from the Hessian Matrix of the two loss functions. Experimental results on real-world data, demonstrate that S-GGNs exhibit superior convergence, robustness, and generalization, compared with state-of-the-arts.

Lingyu Feng, Ting Gao, Wang Xiao et al.

Detecting early warning indicators for abrupt dynamical transitions in complex systems or high-dimensional observation data is essential in many real-world applications, such as brain diseases, natural disasters, and engineering reliability. To this end, we develop a novel approach: the directed anisotropic diffusion map that captures the latent evolutionary dynamics in the low-dimensional manifold. Then three effective warning signals (Onsager-Machlup Indicator, Sample Entropy Indicator, and Transition Probability Indicator) are derived through the latent coordinates and the latent stochastic dynamical systems. To validate our framework, we apply this methodology to authentic electroencephalogram (EEG) data. We find that our early warning indicators are capable of detecting the tipping point during state transition. This framework not only bridges the latent dynamics with real-world data but also shows the potential ability for automatic labeling on complex high-dimensional time series.

Lingyu Feng, Ting Gao, Min Dai et al.

Multiscale stochastic dynamical systems have been widely adopted to a variety of scientific and engineering problems due to their capability of depicting complex phenomena in many real world applications. This work is devoted to investigating the effective dynamics for slow-fast stochastic dynamical systems. Given observation data on a short-term period satisfying some unknown slow-fast stochastic systems, we propose a novel algorithm including a neural network called Auto-SDE to learn invariant slow manifold. Our approach captures the evolutionary nature of a series of time-dependent autoencoder neural networks with the loss constructed from a discretized stochastic differential equation. Our algorithm is also validated to be accurate, stable and effective through numerical experiments under various evaluation metrics.

Luxuan Yang, Ting Gao, Wei Wei et al.

Time series classification faces two unavoidable problems. One is partial feature information and the other is poor label quality, which may affect model performance. To address the above issues, we create a label correction method to time series data with meta-learning under a multi-task framework. There are three main contributions. First, we train the label correction model with a two-branch neural network in the outer loop. While in the model-agnostic inner loop, we use pre-existing classification models in a multi-task way and jointly update the meta-knowledge so as to help us achieve adaptive labeling on complex time series. Second, we devise new data visualization methods for both image patterns of the historical data and data in the prediction horizon. Finally, we test our method with various financial datasets, including XOM, S\&P500, and SZ50. Results show that our method is more effective and accurate than some existing label correction techniques.

Peng Zhang, Ting Gao, Jin Guo et al.

Early warning for epilepsy patients is crucial for their safety and well-being, in particular to prevent or minimize the severity of seizures. Through the patients' EEG data, we propose a meta learning framework to improve the prediction of early ictal signals. The proposed bi-level optimization framework can help automatically label noisy data at the early ictal stage, as well as optimize the training accuracy of the backbone model. To validate our approach, we conduct a series of experiments to predict seizure onset in various long-term windows, with LSTM and ResNet implemented as the baseline models. Our study demonstrates that not only the ictal prediction accuracy obtained by meta learning is significantly improved, but also the resulting model captures some intrinsic patterns of the noisy data that a single backbone model could not learn. As a result, the predicted probability generated by the meta network serves as a highly effective early warning indicator.

Chunhai Jiao, Jin Guo, Haoyan Zhang et al.

We study the problem of detecting the onset of path splitting in stochastic interpolation between probability distributions. This question is especially subtle when the target distribution is nonconvex or supported on disconnected components, where interpolating trajectories may separate into distinct branches. Motivated by the stochastic control and Schrödinger bridge viewpoint, we propose a Jacobi field based indicator for identifying candidate splitting times and locations. Our approach is based on the Jacobi field associated with the linearization of an induced interpolating flow. Starting from a stochastic interpolation ansatz, we construct an Eulerian velocity field by conditional averaging and derive its spatial Jacobian in terms of the local posterior geometry of the target sample cloud. This allows us to interpret the symmetric part of the Jacobian as a local strain tensor and to use its spectral structure to quantify the amplification of infinitesimal perturbations along reference trajectories. Numerical experiments on non-convex and disconnected target distributions show that the proposed indicator consistently localizes the emergence of branching regions and captures the temporal development of splitting. These results suggest that Jacobi field analysis provides a natural mathematical framework for studying local instability and splitting phenomena in stochastic interpolation.

Longfei Guo, Pengbo Li, Ting Gao et al.

With the rapid advancement of AI technology, we have seen more and more concerns on data privacy, leading to some cutting-edge research on machine learning with encrypted computation. Fully Homomorphic Encryption (FHE) is a crucial technology for privacy-preserving computation, while it struggles with continuous non-polynomial functions, as it operates on discrete integers and supports only addition and multiplication. Spiking Neural Networks (SNNs), which use discrete spike signals, naturally complement FHE's characteristics. In this paper, we introduce FHE-DiCSNN, a framework built on the TFHE scheme, utilizing the discrete nature of SNNs for secure and efficient computations. By leveraging bootstrapping techniques, we successfully implement Leaky Integrate-and-Fire (LIF) neuron models on ciphertexts, allowing SNNs of arbitrary depth. Our framework is adaptable to other spiking neuron models, offering a novel approach to homomorphic evaluation of SNNs. Additionally, we integrate convolutional methods inspired by CNNs to enhance accuracy and reduce the simulation time associated with random encoding. Parallel computation techniques further accelerate bootstrapping operations. Experimental results on the MNIST and FashionMNIST datasets validate the effectiveness of FHE-DiCSNN, with a loss of less than 3\% compared to plaintext, respectively, and computation times of under 1 second per prediction. We also apply the model into real medical image classification problems and analyze the parameter optimization and selection.

Yixin Wang, Ting Gao, Jinqiao Duan

We introduce a framework for analyzing topological tipping in time-evolutionary point clouds by extending the recently proposed Topological Optimal Transport (TpOT) distance. While TpOT unifies geometric, homological, and higher-order relations into one metric, its global scalar distance can obscure transient, localized structural reorganizations during dynamic phase transitions. To overcome this limitation, we present a hierarchical dynamic evaluation framework driven by a novel topological and hypergraph reconstruction strategy. Instead of directly interpolating abstract network parameters, our method interpolates the underlying spatial geometry and rigorously recomputes the valid topological structures, ensuring physical fidelity. Along this geodesic, we introduce a set of multi-scale indicators: macroscopic metrics (Topological Distortion and Persistence Entropy) to capture global shifts, and a novel mesoscopic dual-perspective Hypergraph Entropy (node-perspective and edge-perspective) to detect highly sensitive, asynchronous local rewirings. We further propagate the cycle-level entropy change onto individual vertices to form a point-level topological field. Extensive evaluations on physical dynamical systems (Rayleigh-Van der Pol limit cycles, Double-Well cluster fusion), high-dimensional biological aggregation (D'Orsogna model), and longitudinal stroke fMRI data demonstrate the utility of combining transport-based alignment with multi-scale entropy diagnostics for dynamic topological analysis.

Yang Li, Shengyuan Xu, Jinqiao Duan

Discovering explicit governing equations of stochastic dynamical systems with both (Gaussian) Brownian noise and (non-Gaussian) Lévy noise from data is chanllenging due to possible intricate functional forms and the inherent complexity of Lévy motion. This present research endeavors to develop an evolutionary symbol sparse regression (ESSR) approach to extract non-Gaussian stochastic dynamical systems from sample path data, based on nonlocal Kramers-Moyal formulas, genetic programming, and sparse regression. More specifically, the genetic programming is employed to generate a diverse array of candidate functions, the sparse regression technique aims at learning the coefficients associated with these candidates, and the nonlocal Kramers-Moyal formulas serve as the foundation for constructing the fitness measure in genetic programming and the loss function in sparse regression. The efficacy and capabilities of this approach are showcased through its application to several illustrative models. This approach stands out as a potent instrument for deciphering non-Gaussian stochastic dynamics from available datasets, indicating a wide range of applications across different fields.

Cheng Fang, Jinqiao Duan

Establishing appropriate mathematical models for complex systems in natural phenomena not only helps deepen our understanding of nature but can also be used for state estimation and prediction. However, the extreme complexity of natural phenomena makes it extremely challenging to develop full-order models (FOMs) and apply them to studying many quantities of interest. In contrast, appropriate reduced-order models (ROMs) are favored due to their high computational efficiency and ability to describe the key dynamics and statistical characteristics of natural phenomena. Taking the viscous Burgers equation as an example, this paper constructs a Convolutional Autoencoder-Reservoir Computing-Normalizing Flow algorithm framework, where the Convolutional Autoencoder is used to construct latent space representations, and the Reservoir Computing-Normalizing Flow framework is used to characterize the evolution of latent state variables. In this way, a data-driven stochastic parameter reduced-order model is constructed to describe the complex system and its dynamic behavior.

Weiguo Lu, Xuan Wu, Deng Ding et al.

Diffusion models (DMs) are a type of generative model that has a huge impact on image synthesis and beyond. They achieve state-of-the-art generation results in various generative tasks. A great diversity of conditioning inputs, such as text or bounding boxes, are accessible to control the generation. In this work, we propose a conditioning mechanism utilizing Gaussian mixture models (GMMs) as feature conditioning to guide the denoising process. Based on set theory, we provide a comprehensive theoretical analysis that shows that conditional latent distribution based on features and classes is significantly different, so that conditional latent distribution on features produces fewer defect generations than conditioning on classes. Two diffusion models conditioned on the Gaussian mixture model are trained separately for comparison. Experiments support our findings. A novel gradient function called the negative Gaussian mixture gradient (NGMG) is proposed and applied in diffusion model training with an additional classifier. Training stability has improved. We also theoretically prove that NGMG shares the same benefit as the Earth Mover distance (Wasserstein) as a more sensible cost function when learning distributions supported by low-dimensional manifolds.

Wei Wei, Ting Gao, Jinqiao Duan et al.

Many complex real world phenomena exhibit abrupt, intermittent or jumping behaviors, which are more suitable to be described by stochastic differential equations under non-Gaussian Lévy noise. Among these complex phenomena, the most likely transition paths between metastable states are important since these rare events may have a high impact in certain scenarios. Based on the large deviation principle, the most likely transition path could be treated as the minimizer of the rate function upon paths that connect two points. One of the challenges to calculate the most likely transition path for stochastic dynamical systems under non-Gaussian Lévy noise is that the associated rate function can not be explicitly expressed by paths. For this reason, we formulate an optimal control problem to obtain the optimal state as the most likely transition path. We then develop a neural network method to solve this issue. Several experiments are investigated for both Gaussian and non-Gaussian cases.

Cheng Fang, Yubin Lu, Ting Gao et al.

Recently, extracting data-driven governing laws of dynamical systems through deep learning frameworks has gained a lot of attention in various fields. Moreover, a growing amount of research work tends to transfer deterministic dynamical systems to stochastic dynamical systems, especially those driven by non-Gaussian multiplicative noise. However, lots of log-likelihood based algorithms that work well for Gaussian cases cannot be directly extended to non-Gaussian scenarios which could have high error and low convergence issues. In this work, we overcome some of these challenges and identify stochastic dynamical systems driven by $α$-stable Lévy noise from only random pairwise data. Our innovations include: (1) designing a deep learning approach to learn both drift and diffusion coefficients for Lévy induced noise with $α$ across all values, (2) learning complex multiplicative noise without restrictions on small noise intensity, (3) proposing an end-to-end complete framework for stochastic systems identification under a general input data assumption, that is, $α$-stable random variable. Finally, numerical experiments and comparisons with the non-local Kramers-Moyal formulas with moment generating function confirm the effectiveness of our method.

Luxuan Yang, Ting Gao, Yubin Lu et al.

In this article, we employ a collection of stochastic differential equations with drift and diffusion coefficients approximated by neural networks to predict the trend of chaotic time series which has big jump properties. Our contributions are, first, we propose a model called Lévy induced stochastic differential equation network, which explores compounded stochastic differential equations with $α$-stable Lévy motion to model complex time series data and solve the problem through neural network approximation. Second, we theoretically prove that the numerical solution through our algorithm converges in probability to the solution of corresponding stochastic differential equation, without curse of dimensionality. Finally, we illustrate our method by applying it to real financial time series data and find the accuracy increases through the use of non-Gaussian Lévy processes. We also present detailed comparisons in terms of data patterns, various models, different shapes of Lévy motion and the prediction lengths.

Yang Li, Yubin Lu, Shengyuan Xu et al.

With the rapid increase of valuable observational, experimental and simulated data for complex systems, much efforts have been devoted to identifying governing laws underlying the evolution of these systems. Despite the wide applications of non-Gaussian fluctuations in numerous physical phenomena, the data-driven approaches to extract stochastic dynamical systems with (non-Gaussian) Lévy noise are relatively few so far. In this work, we propose a data-driven method to extract stochastic dynamical systems with $α$-stable Lévy noise from short burst data based on the properties of $α$-stable distributions. More specifically, we first estimate the Lévy jump measure and noise intensity via computing mean and variance of the amplitude of the increment of the sample paths. Then we approximate the drift coefficient by combining nonlocal Kramers-Moyal formulas with normalizing flows. Numerical experiments on one- and two-dimensional prototypical examples illustrate the accuracy and effectiveness of our method. This approach will become an effective scientific tool in discovering stochastic governing laws of complex phenomena and understanding dynamical behaviors under non-Gaussian fluctuations.

Yubin Lu, Yang Li, Jinqiao Duan

With the rapid development of computational techniques and scientific tools, great progress of data-driven analysis has been made to extract governing laws of dynamical systems from data. Despite the wide occurrences of non-Gaussian fluctuations, the effective data-driven methods to identify stochastic differential equations with non-Gaussian Lévy noise are relatively few so far. In this work, we propose a data-driven approach to extract stochastic governing laws with both (Gaussian) Brownian motion and (non-Gaussian) Lévy motion, from short bursts of simulation data. Specifically, we use the normalizing flows technology to estimate the transition probability density function (solution of nonlocal Fokker-Planck equation) from data, and then substitute it into the recently proposed nonlocal Kramers-Moyal formulas to approximate Lévy jump measure, drift coefficient and diffusion coefficient. We demonstrate that this approach can learn the stochastic differential equation with Lévy motion. We present examples with one- and two-dimensional, decoupled and coupled systems to illustrate our method. This approach will become an effective tool for discovering stochastic governing laws and understanding complex dynamical behaviors.

Yubin Lu, Romit Maulik, Ting Gao et al.

In this work, we propose a method to learn multivariate probability distributions using sample path data from stochastic differential equations. Specifically, we consider temporally evolving probability distributions (e.g., those produced by integrating local or nonlocal Fokker-Planck equations). We analyze this evolution through machine learning assisted construction of a time-dependent mapping that takes a reference distribution (say, a Gaussian) to each and every instance of our evolving distribution. If the reference distribution is the initial condition of a Fokker-Planck equation, what we learn is the time-T map of the corresponding solution. Specifically, the learned map is a multivariate normalizing flow that deforms the support of the reference density to the support of each and every density snapshot in time. We demonstrate that this approach can approximate probability density function evolutions in time from observed sampled data for systems driven by both Brownian and Lévy noise. We present examples with two- and three-dimensional, uni- and multimodal distributions to validate the method.

Yang Li, Jinqiao Duan

Advances in data science are leading to new progresses in the analysis and understanding of complex dynamics for systems with experimental and observational data. With numerous physical phenomena exhibiting bursting, flights, hopping, and intermittent features, stochastic differential equations with non-Gaussian Lévy noise are suitable to model these systems. Thus it is desirable and essential to infer such equations from available data to reasonably predict dynamical behaviors. In this work, we consider a data-driven method to extract stochastic dynamical systems with non-Gaussian asymmetric (rather than the symmetric) Lévy process, as well as Gaussian Brownian motion. We establish a theoretical framework and design a numerical algorithm to compute the asymmetric Lévy jump measure, drift and diffusion (i.e., nonlocal Kramers-Moyal formulas), hence obtaining the stochastic governing law, from noisy data. Numerical experiments on several prototypical examples confirm the efficacy and accuracy of this method. This method will become an effective tool in discovering the governing laws from available data sets and in understanding the mechanisms underlying complex random phenomena.

Yang Li, Jinqiao Duan, Xianbin Liu

The emergence of transition phenomena between metastable states induced by noise plays a fundamental role in a broad range of nonlinear systems. The computation of the most probable paths is a key issue to understand the mechanism of transition behaviors. Shooting method is a common technique for this purpose to solve the Euler-Lagrange equation for the associated action functional, while losing its efficacy in high-dimensional systems. In the present work, we develop a machine learning framework to compute the most probable paths in the sense of Onsager-Machlup action functional theory. Specifically, we reformulate the boundary value problem of Hamiltonian system and design a neural network to remedy the shortcomings of shooting method. The successful applications of our algorithms to several prototypical examples demonstrate its efficacy and accuracy for stochastic systems with both (Gaussian) Brownian noise and (non-Gaussian) Lévy noise. This novel approach is effective in exploring the internal mechanisms of rare events triggered by random fluctuations in various scientific fields.

Xiaoli Chen, Liu Yang, Jinqiao Duan et al.

The Fokker-Planck (FP) equation governing the evolution of the probability density function (PDF) is applicable to many disciplines but it requires specification of the coefficients for each case, which can be functions of space-time and not just constants, hence requiring the development of a data-driven modeling approach. When the data available is directly on the PDF, then there exist methods for inverse problems that can be employed to infer the coefficients and thus determine the FP equation and subsequently obtain its solution. Herein, we address a more realistic scenario, where only sparse data are given on the particles' positions at a few time instants, which are not sufficient to accurately construct directly the PDF even at those times from existing methods, e.g., kernel estimation algorithms. To this end, we develop a general framework based on physics-informed neural networks (PINNs) that introduces a new loss function using the Kullback-Leibler divergence to connect the stochastic samples with the FP equation, to simultaneously learn the equation and infer the multi-dimensional PDF at all times. In particular, we consider two types of inverse problems, type I where the FP equation is known but the initial PDF is unknown, and type II in which, in addition to unknown initial PDF, the drift and diffusion terms are also unknown. In both cases, we investigate problems with either Brownian or Levy noise or a combination of both. We demonstrate the new PINN framework in detail in the one-dimensional case (1D) but we also provide results for up to 5D demonstrating that we can infer both the FP equation and} dynamics simultaneously at all times with high accuracy using only very few discrete observations of the particles.

Yang Li, Jinqiao Duan

With the rapid increase of valuable observational, experimental and simulating data for complex systems, great efforts are being devoted to discovering governing laws underlying the evolution of these systems. However, the existing techniques are limited to extract governing laws from data as either deterministic differential equations or stochastic differential equations with Gaussian noise. In the present work, we develop a new data-driven approach to extract stochastic dynamical systems with non-Gaussian symmetric Lévy noise, as well as Gaussian noise. First, we establish a feasible theoretical framework, by expressing the drift coefficient, diffusion coefficient and jump measure (i.e., anomalous diffusion) for the underlying stochastic dynamical system in terms of sample paths data. We then design a numerical algorithm to compute the drift, diffusion coefficient and jump measure, and thus extract a governing stochastic differential equation with Gaussian and non-Gaussian noise. Finally, we demonstrate the efficacy and accuracy of our approach by applying to several prototypical one-, two- and three-dimensional systems. This new approach will become a tool in discovering governing dynamical laws from noisy data sets, from observing or simulating complex phenomena, such as rare events triggered by random fluctuations with heavy as well as light tail statistical features.

Xu Sun, Jinqiao Duan, Xiaofan Li et al.

The Kalman filter is extensively used for state estimation for linear systems under Gaussian noise. When non-Gaussian Lévy noise is present, the conventional Kalman filter may fail to be effective due to the fact that the non-Gaussian Lévy noise may have infinite variance. A modified Kalman filter for linear systems with non-Gaussian Lévy noise is devised. It works effectively with reasonable computational cost. Simulation results are presented to illustrate this non-Gaussian filtering method.

Jiarui Yang, Jinqiao Duan

Uncertainties are abundant in complex systems. Mathematical models for these systems thus contain random effects or noises. The models are often in the form of stochastic differential equations, with some parameters to be determined by observations. The stochastic differential equations may be driven by Brownian motion, fractional Brownian motion or Lévy motion. After a brief overview of recent advances in estimating parameters in stochastic differential equations, various numerical algorithms for computing parameters are implemented. The numerical simulation results are shown to be consistent with theoretical analysis. Moreover, for fractional Brownian motion and $α-$stable Lévy motion, several algorithms are reviewed and implemented to numerically estimate the Hurst parameter $H$ and characteristic exponent $α$.

Manfred Denker, Jinqiao Duan, Michael McCourt

We propose a new algorithm for generating pseudorandom (pseudo-generic) numbers of conformal measures of a continuous map T acting on a compact space X and for a Holder continuous potential F. In particular, we show that this algorithm provides good approximations to generic points for hyperbolic rational functions of degree two and the potential -h log|T'|, where h denotes the Hausdorff dimension of the Julia set of T .