Lizuo Liu

Keenan Eikenberry, Lizuo Liu, Yoonsang Lee

This work develops a framework for post-training augmentation invariance, in which our goal is to add invariance properties to a pretrained network without altering its behavior on the original, non-augmented input distribution. We define this notion precisely and additionally introduce augmented encoders, which are probabilistic encoders that formalize augmentation-based encoding processes and that serve as our fundamental object of study. We introduce two losses for augmented encoders, namely, Markov-Wasserstein minimization and Wasserstein correlation maximization, and we demonstrate empirically that both losses can be used to train lightweight, one-hidden-layer MLP adapter networks E_theta that, when appended to the latent space of a pretrained network F, do indeed lead to (approximate) post-training augmentation invariance. For example, on STL10 with F = DINOv2 features, the composite network C o E_theta o F, where C is a linear classifier and where E_theta is one of our proposed adapter networks, achieves 94% classification accuracy on arbitrarily rotated images, whereas a network of the form C o F without the adapter E_theta drops to 71% accuracy. Similarly, we can boost noise-invariant classification results from 58% up to 86%. Significantly, we obtain these results with no fine-tuning (the weights of F remain frozen throughout), and our methods introduce little corruption to the original features, since E_theta acts nearly isometrically on the non-augmented latent distribution. In contrast, we show that adapter networks trained with alternative candidate losses, specifically SimCLR and HSIC maximization, produce uncompetitive classification results and fundamentally corrupt the original latent space. Code available at: https://github.com/keenan-eikenberry/augmentation_invariance

Lizuo Liu, Lu Zhang, Anne Gelb

We propose a neural entropy-stable conservative flux form neural network (NESCFN) for learning hyperbolic conservation laws and their associated entropy functions directly from solution trajectories, without requiring any predefined numerical discretization. While recent neural network architectures have successfully integrated classical numerical principles into learned models, most rely on prior knowledge of the governing equations or assume a fixed discretization. Our approach removes this dependency by embedding entropy-stable design principles into the learning process itself, enabling the discovery of physically consistent dynamics in a fully data-driven setting. By jointly learning both the numerical flux function and a corresponding entropy, the proposed method ensures conservation and entropy dissipation, critical for long-term stability and fidelity in the system of hyperbolic conservation laws. Numerical results demonstrate that the method achieves stability and conservation over extended time horizons and accurately captures shock propagation speeds, even without oracle access to future-time solution profiles in the training data.

Bo Wang, Lizuo Liu, Wei Cai

In this paper, a multi-scale DeepOnet (Mscale-DeepOnet) is proposed to reduce the spectral bias of the DeepOnet in learning high-frequency mapping between highly oscillatory functions, with an application to the nonlinear mapping between the coefficient of the Helmholtz equation and its solution. The Mscale-DeepOnet introduces the multiscale neural network in the branch and trunk networks of the original DeepOnet, the resulting Mscale-DeepOnet is shown to be able to capture various high-frequency components of the mapping itself and its image. Numerical results demonstrate the substantial improvement of the Mscale-DeepOnet for the problem of wave scattering in the high-frequency regime over the normal DeepOnet with a similar number of network parameters.

Lizuo Liu, Tongtong Li, Anne Gelb et al.

We propose an entropy-stable conservative flux form neural network (CFN) that integrates classical numerical conservation laws into a data-driven framework using the entropy-stable, second-order, and non-oscillatory Kurganov-Tadmor (KT) scheme. The proposed entropy-stable CFN uses slope limiting as a denoising mechanism, ensuring accurate predictions in both noisy and sparse observation environments, as well as in both smooth and discontinuous regions. Numerical experiments demonstrate that the entropy-stable CFN achieves both stability and conservation while maintaining accuracy over extended time domains. Furthermore, it successfully predicts shock propagation speeds in long-term simulations, {\it without} oracle knowledge of later-time profiles in the training data.

Lizuo Liu, Lu Zhang, Anne Gelb

We propose a parametric hyperbolic conservation law (SymCLaw) for learning hyperbolic systems directly from data while ensuring conservation, entropy stability, and hyperbolicity by design. Unlike existing approaches that typically enforce only conservation or rely on prior knowledge of the governing equations, our method parameterizes the flux functions in a form that guarantees real eigenvalues and complete eigenvectors of the flux Jacobian, thereby preserving hyperbolicity. At the same time, we embed entropy-stable design principles by jointly learning a convex entropy function and its associated flux potential, ensuring entropy dissipation and the selection of physically admissible weak solutions. A corresponding entropy-stable numerical flux scheme provides compatibility with standard discretizations, allowing seamless integration into classical solvers. Numerical experiments on benchmark problems, including Burgers, shallow water, Euler, and KPP equations, demonstrate that SymCLaw generalizes to unseen initial conditions, maintains stability under noisy training data, and achieves accurate long-time predictions, highlighting its potential as a principled foundation for data-driven modeling of hyperbolic conservation laws.

Wei Cai, Xiaoguang Li, Lizuo Liu

In this paper, we propose a phase shift deep neural network (PhaseDNN), which provides a uniform wideband convergence in approximating high frequency functions and solutions of wave equations. The PhaseDNN makes use of the fact that common DNNs often achieve convergence in the low frequency range first, and a series of moderately-sized DNNs are constructed and trained for selected high frequency ranges. With the help of phase shifts in the frequency domain, each of the DNNs will be trained to approximate the function's higher frequency content over a specific range at the the speed of convergence as in the low frequency range. As a result, the proposed PhaseDNN is able to convert high frequency learning to low frequency one, allowing a uniform learning to wideband functions. The PhaseDNN will then be applied to find the solution of high frequency wave equations in inhomogeneous media through both differential and integral equation formulations with least square residual loss functions. Numerical results have demonstrated the capability of the PhaseDNN in learning high frequency functions and oscillatory solutions of interior and exterior Helmholtz equations.

Wei Cai, Xiaoguang Li, Lizuo Liu

In this paper, we propose a phase shift deep neural network (PhaseDNN) which provides a wideband convergence in approximating a high dimensional function during its training of the network. The PhaseDNN utilizes the fact that many DNN achieves convergence in the low frequency range first, thus, a series of moderately-sized of DNNs are constructed and trained in parallel for ranges of higher frequencies. With the help of phase shifts in the frequency domain, implemented through a simple phase factor multiplication on the training data, each DNN in the series will be trained to approximate the target function's higher frequency content over a specific range. Due to the phase shift, each DNN achieves the speed of convergence as in the low frequency range. As a result, the proposed PhaseDNN system is able to convert wideband frequency learning to low frequency learning, thus allowing a uniform learning to wideband high dimensional functions with frequency adaptive training. Numerical results have demonstrated the capability of PhaseDNN in learning information of a target function from low to high frequency uniformly.