Siu Wun Cheung

Maria Vasilyeva, Eric T. Chung, Siu Wun Cheung et al.

Our goal of this paper is to develop a new upscaling method for multicontinua flow problems in fractured porous media. We consider a system of equations that describes flow phenomena with multiple flow variables defined on both matrix and fractures. To construct our upscaled model, we will apply the nonlocal multicontinua (NLMC) upscaling technique. The upscaled coefficients are obtained by using some multiscale basis functions, which are solutions of local problems defined on oversampled regions. For each continuum within a target coarse element, we will solve a local problem defined on an oversampling region obtained by extending the target element by few coarse grid layers, with a set of constraints which enforce the local solution to have mean value one on the chosen continuum and zero mean otherwise. The resulting multiscale basis functions have been shown to have good approximation properties. To illustrate the idea of our approach, we will consider a dual continua background model consisting of discrete fractures in two space dimensions, that is, we consider a system with three continua. We will present several numerical examples, and they show that our method is able to capture the interaction between matrix continua and discrete fractures on the coarse grid efficiently.

Siu Wun Cheung, Eric T. Chung, Yalchin Efendiev et al.

The dual continuum model serves as a powerful tool in the modeling of subsurface applications. It allows a systematic coupling of various components of the solutions. The system is of multiscale nature as it involves high heterogeneous and high contrast coefficients. To numerically compute the solutions, some types of reduced order methods are necessary. We will develop and analyze a novel multiscale method based on the recent advances in multiscale finite element methods. Our method will compute multiple local multiscale basis functions per coarse region. The idea is based on some local spectral problems, which are important to identify high contrast channels, and an energy minimization principle. Using these concepts, we show that the basis functions are localized, even in the presence of high contrast long channels and fractures. In addition, we show that the convergence of the method depends only on the coarse mesh size. Finally, we present several numerical tests to show the performance.

Min Wang, Siu Wun Cheung, Eric T. Chung et al.

Simulating flow in a highly heterogeneous reservoir with multiscale characteristics could be considerably demanding. To tackle this problem, we propose a numerical scheme coupling the Generalized Multiscale Finite Element Method (GMsFEM) with a triple-continuum model aimed at a faster simulator framework that can explicitly represent the interactions among different continua. To further enrich the descriptive ability of our proposed model, we combine the Discrete Fracture Model (DFM) to model the local effects of discrete fractures. In the proposed model, GMsFEM, as an advanced model reduction technique, enables capturing the multiscale flow dynamics. This is accomplished by systematically generating an approximation space through solving a series of local snapshot and spectral problems. The resulting eigenfunctions can pass the local features to the global level when acting as basis functions in coarse problems. Our goal in this paper is to further improve the accuracy of flow simulation in complicated reservoirs especially for the case when multiple discrete fractures located in single coarse neighborhood and multiscale finite element methods fail. Together with a detailed description of the model, several numerical experiments are conducted to confirm the success of our proposed method. A rigid proof is also given in the aspect of numerical analysis.

Min Wang, Siu Wun Cheung, Eric T. Chung et al.

In this paper, we propose a deep-learning-based approach to a class of multiscale problems. THe Generalized Multiscale Finite Element Method (GMsFEM) has been proven successful as a model reduction technique of flow problems in heterogeneous and high-contrast porous media. The key ingredients of GMsFEM include mutlsicale basis functions and coarse-scale parameters, which are obtained from solving local problems in each coarse neighborhood. Given a fixed medium, these quantities are precomputed by solving local problems in an offline stage, and result in a reduced-order model. However, these quantities have to be re-computed in case of varying media. The objective of our work is to make use of deep learning techniques to mimic the nonlinear relation between the permeability field and the GMsFEM discretizations, and use neural networks to perform fast computation of GMsFEM ingredients repeatedly for a class of media. We provide numerical experiments to investigate the predictive power of neural networks and the usefulness of the resultant multiscale model in solving channelized porous media flow problems.

Y. Efendiev, W. T. Leung, S. W. Cheung et al.

In this paper, we develop a Bayesian multiscale approach based on a multiscale finite element method. Because of scale disparity in many multiscale applications, computational models can not resolve all scales. Various subgrid models are proposed to represent un-resolved scales. Here, we consider a probabilistic approach for modeling un-resolved scales using the Multiscale Finite Element Method (cf., [1, 2]). By representing dominant modes using the Generalized Multiscale Finite Element, we propose a Bayesian framework, which provides multiple inexpensive (computable) solutions for a deterministic problem. These approximate probabilistic solutions may not be very close to the exact solutions and, thus, many realizations are needed. In this way, we obtain a rigorous probabilistic description of approximate solutions. In the paper, we consider parabolic and wave equations in heterogeneous media. In each time interval, the domain is divided into subregions. Using residual information, we design appropriate prior and posterior distributions. The likelihood consists of the residual minimization. To sample from the resulting posterior distribution, we consider several sampling strategies. The sampling involves identifying important regions and important degrees of freedom beyond permanent basis functions, which are used in residual computation. Numerical results are presented. We consider two sampling algorithms. The first algorithm uses sequential sampling and is inexpensive. In the second algorithm, we perform full sampling using the Gibbs sampling algorithm, which is more accurate compared to the sequential sampling. The main novel ingredients of our approach consist of: defining appropriate permanent basis functions and the corresponding residual; setting up a proper posterior distribution; and sampling the posteriors.

Siu Wun Cheung, Eric Chung, Hyea Hyun Kim

In this paper, we develop a new mass conservative numerical scheme for the simulations of a class of fluid-structure interaction problems. We will use the immersed boundary method to model the fluid-structure interaction, while the fluid flow is governed by the incompressible Navier-Stokes equations. The immersed boundary method is proven to be a successful scheme to model fluid-structure interactions. To ensure mass conservation, we will use the staggered discontinuous Galerkin method to discretize the incompressible Navier-Stokes equations. The staggered discontinuous Galerkin method is able to preserve the skew-symmetry of the convection term. In addition, by using a local postprocessing technique, the weakly divergence free velocity can be used to compute a new postprocessed velocity, which is exactly divergence free and has a superconvergence property. This strongly divergence free velocity field is the key to the mass conservation. Furthermore, energy stability is improved by the skew-symmetric discretization of the convection term. We will present several numerical results to show the performance of the method.

Siu Wun Cheung, Eric T. Chung

In this paper, we present an embedded staggered discontinuous Galerkin method for the convection-diffusion equation. The new method combines the advantages of staggered discontinuous Galerkin (SDG) and embedded discontinuous Galerkin (EDG) method, and results in many good properties, namely local and global conservations, free of carefully designed stabilization terms or flux conditions and high computational efficiency. In applying the new method to convection-dominated problems, the method provides optimal convergence in potential and suboptimal convergence in flux, which is comparable to other existing DG methods, and achieves $L^2$ stability by making use of a skew-symmetric discretization of the convection term, irrespective of diffusivity. We will present numerical results to show the performance of the method.

Siu Wun Cheung, Nilabja Guha

In this paper, we propose a Bayesian approach for multiscale problems with the availability of dynamic observational data. Our method selects important degrees of freedom probabilistically in a Generalized multiscale finite element method framework. Due to scale disparity in many multiscale applications, computational models can not resolve all scales. Dominant modes in the Generalized Multiscale Finite Element Method are used as "permanent" basis functions, which we use to compute an inexpensive multiscale solution and the associated uncertainties. Through our Bayesian framework, we can model approximate solutions by selecting the unresolved scales probabilistically. We consider parabolic equations in heterogeneous media. The temporal domain is partitioned into subintervals. Using residual information and given dynamic data, we design appropriate prior distribution for modeling missing subgrid information. The likelihood is designed to minimize the residual in the underlying PDE problem and the mismatch of observational data. Using the resultant posterior distribution, the sampling process identifies important degrees of freedom beyond permanent basis functions. The method adds important degrees of freedom in resolving subgrid information and ensuring the accuracy of the observations.

Christophe Bonneville, Xiaolong He, April Tran et al.

Numerical solvers of partial differential equations (PDEs) have been widely employed for simulating physical systems. However, the computational cost remains a major bottleneck in various scientific and engineering applications, which has motivated the development of reduced-order models (ROMs). Recently, machine-learning-based ROMs have gained significant popularity and are promising for addressing some limitations of traditional ROM methods, especially for advection dominated systems. In this chapter, we focus on a particular framework known as Latent Space Dynamics Identification (LaSDI), which transforms the high-fidelity data, governed by a PDE, to simpler and low-dimensional latent-space data, governed by ordinary differential equations (ODEs). These ODEs can be learned and subsequently interpolated to make ROM predictions. Each building block of LaSDI can be easily modulated depending on the application, which makes the LaSDI framework highly flexible. In particular, we present strategies to enforce the laws of thermodynamics into LaSDI models (tLaSDI), enhance robustness in the presence of noise through the weak form (WLaSDI), select high-fidelity training data efficiently through active learning (gLaSDI, GPLaSDI), and quantify the ROM prediction uncertainty through Gaussian processes (GPLaSDI). We demonstrate the performance of different LaSDI approaches on Burgers equation, a non-linear heat conduction problem, and a plasma physics problem, showing that LaSDI algorithms can achieve relative errors of less than a few percent and up to thousands of times speed-ups.

Jun Sur Richard Park, Siu Wun Cheung, Youngsoo Choi et al.

We propose a latent space dynamics identification method, namely tLaSDI, that embeds the first and second principles of thermodynamics. The latent variables are learned through an autoencoder as a nonlinear dimension reduction model. The latent dynamics are constructed by a neural network-based model that precisely preserves certain structures for the thermodynamic laws through the GENERIC formalism. An abstract error estimate is established, which provides a new loss formulation involving the Jacobian computation of autoencoder. The autoencoder and the latent dynamics are simultaneously trained to minimize the new loss. Computational examples demonstrate the effectiveness of tLaSDI, which exhibits robust generalization ability, even in extrapolation. In addition, an intriguing correlation is empirically observed between a quantity from tLaSDI in the latent space and the behaviors of the full-state solution.

Youngsoo Choi, Siu Wun Cheung, Youngkyu Kim et al.

The widespread success of foundation models in natural language processing and computer vision has inspired researchers to extend the concept to scientific machine learning and computational science. However, this position paper argues that as the term "foundation model" is an evolving concept, its application in computational science is increasingly used without a universally accepted definition, potentially creating confusion and diluting its precise scientific meaning. In this paper, we address this gap by proposing a formal definition of foundation models in computational science, grounded in the core values of generality, reusability, and scalability. We articulate a set of essential and desirable characteristics that such models must exhibit, drawing parallels with traditional foundational methods, like the finite element and finite volume methods. Furthermore, we introduce the Data-Driven Finite Element Method (DD-FEM), a framework that fuses the modular structure of classical FEM with the representational power of data-driven learning. We demonstrate how DD-FEM addresses many of the key challenges in realizing foundation models for computational science, including scalability, adaptability, and physics consistency. By bridging traditional numerical methods with modern AI paradigms, this work provides a rigorous foundation for evaluating and developing novel approaches toward future foundation models in computational science.

Qing Wan, Siu Wun Cheung, Yoonsuck Choe

Adjoint operators have been found to be effective in the exploration of CNN's inner workings [1]. However, the previous no-bias assumption restricted its generalization. We overcome the restriction via embedding input images into an extended normed space that includes bias in all CNN layers as part of the extended space and propose an adjoint-operator-based algorithm that maps high-level weights back to the extended input space for reconstructing an effective hypersurface. Such hypersurface can be computed for an arbitrary unit in the CNN, and we prove that this reconstructed hypersurface, when multiplied by the original input (through an inner product), will precisely replicate the output value of each unit. We show experimental results based on the CIFAR-10 and CIFAR-100 data sets where the proposed approach achieves near 0 activation value reconstruction error.

Yating Wang, Siu Wun Cheung, Eric T. Chung et al.

The objective of this paper is to design novel multi-layer neural network architectures for multiscale simulations of flows taking into account the observed data and physical modeling concepts. Our approaches use deep learning concepts combined with local multiscale model reduction methodologies to predict flow dynamics. Using reduced-order model concepts is important for constructing robust deep learning architectures since the reduced-order models provide fewer degrees of freedom. Flow dynamics can be thought of as multi-layer networks. More precisely, the solution (e.g., pressures and saturations) at the time instant $n+1$ depends on the solution at the time instant $n$ and input parameters, such as permeability fields, forcing terms, and initial conditions. One can regard the solution as a multi-layer network, where each layer, in general, is a nonlinear forward map and the number of layers relates to the internal time steps. We will rely on rigorous model reduction concepts to define unknowns and connections for each layer. In each layer, our reduced-order models will provide a forward map, which will be modified ("trained") using available data. It is critical to use reduced-order models for this purpose, which will identify the regions of influence and the appropriate number of variables. Because of the lack of available data, the training will be supplemented with computational data as needed and the interpolation between data-rich and data-deficient models. We will also use deep learning algorithms to train the elements of the reduced model discrete system. We will present main ingredients of our approach and numerical results. Numerical results show that using deep learning and multiscale models, we can improve the forward models, which are conditioned to the available data.