Gaohang Chen

Yongqiang Cai, Gaohang Chen, Zhonghua Qiao

The universal approximation property is fundamental to the success of neural networks, and has traditionally been achieved by training networks without any constraints on their parameters. However, recent experimental research proposed a novel permutation-based training method, which exhibited a desired classification performance without modifying the exact weight values. In this paper, we provide a theoretical guarantee of this permutation training method by proving its ability to guide a ReLU network to approximate one-dimensional continuous functions. Our numerical results further validate this method's efficiency in regression tasks with various initializations. The notable observations during weight permutation suggest that permutation training can provide an innovative tool for describing network learning behavior.

Chaoyu Liu, Zhonghao Li, Gaohang Chen et al.

Neural operators have achieved promising performance on partial differential equations (PDEs), but most existing models are built on fixed Eulerian coordinates. This mismatch between evolving physical structures and static coordinates creates spatial misalignment, leading to unnecessarily non-local operator mappings and reinforcing a smoothness preference near sharp transitions. Inspired by adaptive coordinate transformations in classical PDE analysis, we propose the Adaptive Coordinate Transform (ACT) block, a plug-and-play module for data-driven geometric adaptation in neural operators. ACT blocks resolve this structural limitation by learning adaptive coordinate systems within the operator learning pipeline. Specifically, given an input feature, the ACT block learns a coordinate transformation and represents the same feature under the transformed coordinates via differentiable sampling. This operation preserves the underlying signal while changing its spatial representation, equivalent to expressing the same physical quantity in different coordinate systems. By adapting the coordinate system to the data, ACT allows the network to better track evolving structures, reduce operator complexity, and dynamically focus on critical features to improve learning. We evaluate the proposed approach across diverse PDE benchmarks and multiple neural operator architectures. Experimental results demonstrate consistent and significant improvements in predictive accuracy, indicating that learning coordinate systems provides a powerful mechanism for enhancing operator learning.

Gaohang Chen, Lili Ju, Zhonghua Qiao

Solving partial differential equations (PDEs) with neural networks (NNs) has shown great potential in various scientific and engineering fields. However, most existing NN solvers mainly focus on satisfying the given PDE formulas in the strong or weak sense, without explicitly considering some intrinsic physical properties, such as mass and momentum conservation, or energy dissipation. This limitation may result in nonphysical or unstable numerical solutions, particularly in long-term simulations. To address this issue, we propose ``Sidecar'', a novel framework that enhances the physical consistency of existing NN solvers for solving parabolic PDEs. Inspired by the time-dependent spectral renormalization approach, our Sidecar framework introduces a small network as a copilot, guiding the primary function-learning NN solver to respect the structure-preserving properties. Our framework is highly flexible, allowing the preservation of various physical quantities for different PDEs to be incorporated into a wide range of NN solvers. Experimental results on some benchmark problems demonstrate significant improvements brought by the proposed framework to both accuracy and structure preservation of existing NN solvers.