Chuqi Chen

Chuqi Chen, Yue Wu, Yang Xiang

In this paper, we propose a novel approach to generative modeling using a loss function based on elastic interaction energy (EIE), which is inspired by the elastic interaction between defects in crystals. The utilization of the EIE-based metric presents several advantages, including its long range property that enables consideration of global information in the distribution. Moreover, its inclusion of a self-interaction term helps to prevent mode collapse and captures all modes of distribution. To overcome the difficulty of the relatively scattered distribution of high-dimensional data, we first map the data into a latent feature space and approximate the feature distribution instead of the data distribution. We adopt the GAN framework and replace the discriminator with a feature transformation network to map the data into a latent space. We also add a stabilizing term to the loss of the feature transformation network, which effectively addresses the issue of unstable training in GAN-based algorithms. Experimental results on popular datasets, such as MNIST, FashionMNIST, CIFAR-10, and CelebA, demonstrate that our EIEG GAN model can mitigate mode collapse, enhance stability, and improve model performance.

Chuqi Chen, Yue Wu, Yang Xiang

In this paper, we investigate the training process of generative networks that use a type of probability density distance named particle-based distance as the objective function, e.g. MMD GAN, Cramér GAN, EIEG GAN. However, these GANs often suffer from the problem of unstable training. In this paper, we analyze the stability of the training process of these GANs from the perspective of probability density dynamics. In our framework, we regard the discriminator $D$ in these GANs as a feature transformation mapping that maps high dimensional data into a feature space, while the generator $G$ maps random variables to samples that resemble real data in terms of feature space. This perspective enables us to perform stability analysis for the training of GANs using the Wasserstein gradient flow of the probability density function. We find that the training process of the discriminator is usually unstable due to the formulation of $\min_G \max_D E(G, D)$ in GANs. To address this issue, we add a stabilizing term in the discriminator loss function. We conduct experiments to validate our stability analysis and stabilizing method.

Yue Wu, Tianyu Jin, Chuqi Chen et al.

We propose an energy stable network (EStable-Net) for solving gradient flow equations. The EStable-Net enables decreasing of a discrete energy along the neural network, which is consistent with the property of the gradient flow equation. The architecture of the neural network EStable-Net is based on the block network structure (Autoflow) in which output of each block can be interpreted as an intermediate state of the evolution process of the equation, and the energy stable property is incorporated in each block, which is easily generalized to include other physical and/or numerical properties. Our EStable-Net is a supervised learning network approach for solving evolution equations which does not depend on the convergence of time step goes to 0, and can be applied generally even when only data is available but the equation is unknown. We also propose a training strategy for supervised learning that employs data of the evolution stages with different nature. The EStable-Net is validated by numerical experimental results based on the Allen-Cahn equation and the Cahn-Hilliard equation in two dimensions.

Chuqi Chen, Yahong Yang, Yang Xiang et al.

Neural network-based approaches have recently shown significant promise in solving partial differential equations (PDEs) in science and engineering, especially in scenarios featuring complex domains or incorporation of empirical data. One advantage of the neural network methods for PDEs lies in its automatic differentiation (AD), which necessitates only the sample points themselves, unlike traditional finite difference (FD) approximations that require nearby local points to compute derivatives. In this paper, we quantitatively demonstrate the advantage of AD in training neural networks. The concept of truncated entropy is introduced to characterize the training property. Specifically, through comprehensive experimental and theoretical analyses conducted on random feature models and two-layer neural networks, we discover that the defined truncated entropy serves as a reliable metric for quantifying the residual loss of random feature models and the training speed of neural networks for both AD and FD methods. Our experimental and theoretical analyses demonstrate that, from a training perspective, AD outperforms FD in solving PDEs.

Chuqi Chen, Yahong Yang, Yang Xiang et al.

Solving partial differential equations (PDEs) using neural networks has become a central focus in scientific machine learning. Training neural networks for singularly perturbed problems is particularly challenging due to certain parameters in the PDEs that introduce near-singularities in the loss function. In this study, we overcome this challenge by introducing a novel method based on homotopy dynamics to effectively manipulate these parameters. From a theoretical perspective, we analyze the effects of these parameters on training difficulty in these singularly perturbed problems and establish the convergence of the proposed homotopy dynamics method. Experimentally, we demonstrate that our approach significantly accelerates convergence and improves the accuracy of these singularly perturbed problems. These findings present an efficient optimization strategy leveraging homotopy dynamics, offering a robust framework to extend the applicability of neural networks for solving singularly perturbed differential equations.

Chuqi Chen, Yang Xiang, Weihong Zhang

Operator learning has emerged as a powerful paradigm for approximating solution operators of partial differential equations (PDEs) and other functional mappings. \textcolor{red}{}{Classical approaches} typically adopt a pointwise-to-pointwise framework, where input functions are sampled at prescribed locations and mapped directly to solution values. We propose the Fixed-Basis Coefficient to Coefficient Operator Network (FB-C2CNet), which learns operators in the coefficient space induced by prescribed basis functions. In this framework, the input function is projected onto a fixed set of basis functions (e.g., random features or finite element bases), and the neural operator predicts the coefficients of the solution function in the same or another basis. By decoupling basis selection from network training, FB-C2CNet reduces training complexity, enables systematic analysis of how basis choice affects approximation accuracy, and clarifies what properties of coefficient spaces (such as effective rank and coefficient variations) are critical for generalization. Numerical experiments on Darcy flow, Poisson equations in regular, complex, and high-dimensional domains, and elasticity problems demonstrate that FB-C2CNet achieves high accuracy and computational efficiency, showing its strong potential for practical operator learning tasks.