Siu Wun Cheung

Maria Vasilyeva, Eric T. Chung, Siu Wun Cheung et al.

Our goal of this paper is to develop a new upscaling method for multicontinua flow problems in fractured porous media. We consider a system of equations that describes flow phenomena with multiple flow variables defined on both matrix and fractures. To construct our upscaled model, we will apply the nonlocal multicontinua (NLMC) upscaling technique. The upscaled coefficients are obtained by using some multiscale basis functions, which are solutions of local problems defined on oversampled regions. For each continuum within a target coarse element, we will solve a local problem defined on an oversampling region obtained by extending the target element by few coarse grid layers, with a set of constraints which enforce the local solution to have mean value one on the chosen continuum and zero mean otherwise. The resulting multiscale basis functions have been shown to have good approximation properties. To illustrate the idea of our approach, we will consider a dual continua background model consisting of discrete fractures in two space dimensions, that is, we consider a system with three continua. We will present several numerical examples, and they show that our method is able to capture the interaction between matrix continua and discrete fractures on the coarse grid efficiently.

Siu Wun Cheung, Eric T. Chung, Yalchin Efendiev et al.

The dual continuum model serves as a powerful tool in the modeling of subsurface applications. It allows a systematic coupling of various components of the solutions. The system is of multiscale nature as it involves high heterogeneous and high contrast coefficients. To numerically compute the solutions, some types of reduced order methods are necessary. We will develop and analyze a novel multiscale method based on the recent advances in multiscale finite element methods. Our method will compute multiple local multiscale basis functions per coarse region. The idea is based on some local spectral problems, which are important to identify high contrast channels, and an energy minimization principle. Using these concepts, we show that the basis functions are localized, even in the presence of high contrast long channels and fractures. In addition, we show that the convergence of the method depends only on the coarse mesh size. Finally, we present several numerical tests to show the performance.

Min Wang, Siu Wun Cheung, Eric T. Chung et al.

Simulating flow in a highly heterogeneous reservoir with multiscale characteristics could be considerably demanding. To tackle this problem, we propose a numerical scheme coupling the Generalized Multiscale Finite Element Method (GMsFEM) with a triple-continuum model aimed at a faster simulator framework that can explicitly represent the interactions among different continua. To further enrich the descriptive ability of our proposed model, we combine the Discrete Fracture Model (DFM) to model the local effects of discrete fractures. In the proposed model, GMsFEM, as an advanced model reduction technique, enables capturing the multiscale flow dynamics. This is accomplished by systematically generating an approximation space through solving a series of local snapshot and spectral problems. The resulting eigenfunctions can pass the local features to the global level when acting as basis functions in coarse problems. Our goal in this paper is to further improve the accuracy of flow simulation in complicated reservoirs especially for the case when multiple discrete fractures located in single coarse neighborhood and multiscale finite element methods fail. Together with a detailed description of the model, several numerical experiments are conducted to confirm the success of our proposed method. A rigid proof is also given in the aspect of numerical analysis.

Min Wang, Siu Wun Cheung, Eric T. Chung et al.

In this paper, we propose a deep-learning-based approach to a class of multiscale problems. THe Generalized Multiscale Finite Element Method (GMsFEM) has been proven successful as a model reduction technique of flow problems in heterogeneous and high-contrast porous media. The key ingredients of GMsFEM include mutlsicale basis functions and coarse-scale parameters, which are obtained from solving local problems in each coarse neighborhood. Given a fixed medium, these quantities are precomputed by solving local problems in an offline stage, and result in a reduced-order model. However, these quantities have to be re-computed in case of varying media. The objective of our work is to make use of deep learning techniques to mimic the nonlinear relation between the permeability field and the GMsFEM discretizations, and use neural networks to perform fast computation of GMsFEM ingredients repeatedly for a class of media. We provide numerical experiments to investigate the predictive power of neural networks and the usefulness of the resultant multiscale model in solving channelized porous media flow problems.

Siu Wun Cheung, Eric Chung, Hyea Hyun Kim

In this paper, we develop a new mass conservative numerical scheme for the simulations of a class of fluid-structure interaction problems. We will use the immersed boundary method to model the fluid-structure interaction, while the fluid flow is governed by the incompressible Navier-Stokes equations. The immersed boundary method is proven to be a successful scheme to model fluid-structure interactions. To ensure mass conservation, we will use the staggered discontinuous Galerkin method to discretize the incompressible Navier-Stokes equations. The staggered discontinuous Galerkin method is able to preserve the skew-symmetry of the convection term. In addition, by using a local postprocessing technique, the weakly divergence free velocity can be used to compute a new postprocessed velocity, which is exactly divergence free and has a superconvergence property. This strongly divergence free velocity field is the key to the mass conservation. Furthermore, energy stability is improved by the skew-symmetric discretization of the convection term. We will present several numerical results to show the performance of the method.

Siu Wun Cheung, Eric T. Chung

In this paper, we present an embedded staggered discontinuous Galerkin method for the convection-diffusion equation. The new method combines the advantages of staggered discontinuous Galerkin (SDG) and embedded discontinuous Galerkin (EDG) method, and results in many good properties, namely local and global conservations, free of carefully designed stabilization terms or flux conditions and high computational efficiency. In applying the new method to convection-dominated problems, the method provides optimal convergence in potential and suboptimal convergence in flux, which is comparable to other existing DG methods, and achieves $L^2$ stability by making use of a skew-symmetric discretization of the convection term, irrespective of diffusivity. We will present numerical results to show the performance of the method.

Shao-Ting Chiu, Siu Wun Cheung, Ulisses Braga-Neto et al.

Kolmogorov-Arnold Networks (KANs) have shown strong potential for efficiently approximating complex nonlinear functions. However, the original KAN formulation relies on B-spline basis functions, which incur substantial computational overhead due to De Boor's algorithm. To address this limitation, recent work has explored alternative basis functions such as radial basis functions (RBFs) that can improve computational efficiency and flexibility. Yet, standard RBF-KANs often sacrifice accuracy relative to the original KAN design. In this work, we propose Free-RBF-KAN, a RBF-based KAN architecture that incorporates adaptive learning grids and trainable smoothness to close this performance gap. Our method employs freely learnable RBF shapes that dynamically align grid representations with activation patterns, enabling expressive and adaptive function approximation. Additionally, we treat smoothness as a kernel parameter optimized jointly with network weights, without increasing computational complexity. We provide a general universality proof for RBF-KANs, which encompasses our Free-RBF-KAN formulation. Through a broad set of experiments, including multiscale function approximation, physics-informed machine learning, and PDE solution operator learning, Free-RBF-KAN achieves accuracy comparable to the original B-spline-based KAN while delivering faster training and inference. These results highlight Free-RBF-KAN as a compelling balance between computational efficiency and adaptive resolution, particularly for high-dimensional structured modeling tasks.

Siu Wun Cheung, Nilabja Guha

In this paper, we propose a Bayesian approach for multiscale problems with the availability of dynamic observational data. Our method selects important degrees of freedom probabilistically in a Generalized multiscale finite element method framework. Due to scale disparity in many multiscale applications, computational models can not resolve all scales. Dominant modes in the Generalized Multiscale Finite Element Method are used as "permanent" basis functions, which we use to compute an inexpensive multiscale solution and the associated uncertainties. Through our Bayesian framework, we can model approximate solutions by selecting the unresolved scales probabilistically. We consider parabolic equations in heterogeneous media. The temporal domain is partitioned into subintervals. Using residual information and given dynamic data, we design appropriate prior distribution for modeling missing subgrid information. The likelihood is designed to minimize the residual in the underlying PDE problem and the mismatch of observational data. Using the resultant posterior distribution, the sampling process identifies important degrees of freedom beyond permanent basis functions. The method adds important degrees of freedom in resolving subgrid information and ensuring the accuracy of the observations.

Jun Sur Richard Park, Auroni Huque Hashim, Siu Wun Cheung et al.

Data-driven discovery of governing equations from noisy observations remains a fundamental challenge in scientific machine learning. While GENERIC formalism informed neural networks (GFINNs) provide a principled framework that enforces the laws of thermodynamics by construction, their reliance on strong-form loss formulations makes them highly sensitive to measurement noise. To address this limitation, we propose weak formulation-based GENERIC formalism informed neural networks (WGFINNs), which integrate the weak formulation of dynamical systems with the structure-preserving architecture of GFINNs. WGFINNs significantly enhance robustness to noisy data while retaining exact satisfaction of GENERIC degeneracy and symmetry conditions. We further incorporate a state-wise weighted loss and a residual-based attention mechanism to mitigate scale imbalance across state variables. Theoretical analysis contrasts quantitative differences between the strong-form and the weak-form estimators. Mainly, the strong-form estimator diverges as the time step decreases in the presence of noise, while the weak-form estimator can be accurate even with noisy data if test functions satisfy certain conditions. Numerical experiments demonstrate that WGFINNs consistently outperform GFINNs at varying noise levels, achieving more accurate predictions and reliable recovery of physical quantities.

Christophe Bonneville, Xiaolong He, April Tran et al.

Numerical solvers of partial differential equations (PDEs) have been widely employed for simulating physical systems. However, the computational cost remains a major bottleneck in various scientific and engineering applications, which has motivated the development of reduced-order models (ROMs). Recently, machine-learning-based ROMs have gained significant popularity and are promising for addressing some limitations of traditional ROM methods, especially for advection dominated systems. In this chapter, we focus on a particular framework known as Latent Space Dynamics Identification (LaSDI), which transforms the high-fidelity data, governed by a PDE, to simpler and low-dimensional latent-space data, governed by ordinary differential equations (ODEs). These ODEs can be learned and subsequently interpolated to make ROM predictions. Each building block of LaSDI can be easily modulated depending on the application, which makes the LaSDI framework highly flexible. In particular, we present strategies to enforce the laws of thermodynamics into LaSDI models (tLaSDI), enhance robustness in the presence of noise through the weak form (WLaSDI), select high-fidelity training data efficiently through active learning (gLaSDI, GPLaSDI), and quantify the ROM prediction uncertainty through Gaussian processes (GPLaSDI). We demonstrate the performance of different LaSDI approaches on Burgers equation, a non-linear heat conduction problem, and a plasma physics problem, showing that LaSDI algorithms can achieve relative errors of less than a few percent and up to thousands of times speed-ups.

Jun Sur Richard Park, Siu Wun Cheung, Youngsoo Choi et al.

We propose a latent space dynamics identification method, namely tLaSDI, that embeds the first and second principles of thermodynamics. The latent variables are learned through an autoencoder as a nonlinear dimension reduction model. The latent dynamics are constructed by a neural network-based model that precisely preserves certain structures for the thermodynamic laws through the GENERIC formalism. An abstract error estimate is established, which provides a new loss formulation involving the Jacobian computation of autoencoder. The autoencoder and the latent dynamics are simultaneously trained to minimize the new loss. Computational examples demonstrate the effectiveness of tLaSDI, which exhibits robust generalization ability, even in extrapolation. In addition, an intriguing correlation is empirically observed between a quantity from tLaSDI in the latent space and the behaviors of the full-state solution.

Youngsoo Choi, Siu Wun Cheung, Youngkyu Kim et al.

The widespread success of foundation models in natural language processing and computer vision has inspired researchers to extend the concept to scientific machine learning and computational science. However, this position paper argues that as the term "foundation model" is an evolving concept, its application in computational science is increasingly used without a universally accepted definition, potentially creating confusion and diluting its precise scientific meaning. In this paper, we address this gap by proposing a formal definition of foundation models in computational science, grounded in the core values of generality, reusability, and scalability. We articulate a set of essential and desirable characteristics that such models must exhibit, drawing parallels with traditional foundational methods, like the finite element and finite volume methods. Furthermore, we introduce the Data-Driven Finite Element Method (DD-FEM), a framework that fuses the modular structure of classical FEM with the representational power of data-driven learning. We demonstrate how DD-FEM addresses many of the key challenges in realizing foundation models for computational science, including scalability, adaptability, and physics consistency. By bridging traditional numerical methods with modern AI paradigms, this work provides a rigorous foundation for evaluating and developing novel approaches toward future foundation models in computational science.

Qing Wan, Siu Wun Cheung, Yoonsuck Choe

Adjoint operators have been found to be effective in the exploration of CNN's inner workings [1]. However, the previous no-bias assumption restricted its generalization. We overcome the restriction via embedding input images into an extended normed space that includes bias in all CNN layers as part of the extended space and propose an adjoint-operator-based algorithm that maps high-level weights back to the extended input space for reconstructing an effective hypersurface. Such hypersurface can be computed for an arbitrary unit in the CNN, and we prove that this reconstructed hypersurface, when multiplied by the original input (through an inner product), will precisely replicate the output value of each unit. We show experimental results based on the CIFAR-10 and CIFAR-100 data sets where the proposed approach achieves near 0 activation value reconstruction error.

Yating Wang, Siu Wun Cheung, Eric T. Chung et al.

The objective of this paper is to design novel multi-layer neural network architectures for multiscale simulations of flows taking into account the observed data and physical modeling concepts. Our approaches use deep learning concepts combined with local multiscale model reduction methodologies to predict flow dynamics. Using reduced-order model concepts is important for constructing robust deep learning architectures since the reduced-order models provide fewer degrees of freedom. Flow dynamics can be thought of as multi-layer networks. More precisely, the solution (e.g., pressures and saturations) at the time instant $n+1$ depends on the solution at the time instant $n$ and input parameters, such as permeability fields, forcing terms, and initial conditions. One can regard the solution as a multi-layer network, where each layer, in general, is a nonlinear forward map and the number of layers relates to the internal time steps. We will rely on rigorous model reduction concepts to define unknowns and connections for each layer. In each layer, our reduced-order models will provide a forward map, which will be modified ("trained") using available data. It is critical to use reduced-order models for this purpose, which will identify the regions of influence and the appropriate number of variables. Because of the lack of available data, the training will be supplemented with computational data as needed and the interpolation between data-rich and data-deficient models. We will also use deep learning algorithms to train the elements of the reduced model discrete system. We will present main ingredients of our approach and numerical results. Numerical results show that using deep learning and multiscale models, we can improve the forward models, which are conditioned to the available data.