Eric Phipps

Bedřich Sousedík, Roger G. Ghanem, Eric T. Phipps

Use of the stochastic Galerkin finite element methods leads to large systems of linear equations obtained by the discretization of tensor product solution spaces along their spatial and stochastic dimensions. These systems are typically solved iteratively by a Krylov subspace method. We propose a preconditioner which takes an advantage of the recursive hierarchy in the structure of the global matrices. In particular, the matrices posses a recursive hierarchical two-by-two structure, with one of the submatrices block diagonal. Each one of the diagonal blocks in this submatrix is closely related to the deterministic mean-value problem, and the action of its inverse is in the implementation approximated by inner loops of Krylov iterations. Thus our hierarchical Schur complement preconditioner combines, on each level in the approximation of the hierarchical structure of the global matrix, the idea of Schur complement with loops for a number of mutually independent inner Krylov iterations, and several matrix-vector multiplications for the off-diagonal blocks. Neither the global matrix, nor the matrix of the preconditioner need to be formed explicitly. The ingredients include only the number of stiffness matrices from the truncated Karhunen-Loève expansion and a good preconditioned for the mean-value deterministic problem. We provide a condition number bound for a model elliptic problem and the performance of the method is illustrated by numerical experiments.

Saibal De, Zitong Li, Hemanth Kolla et al.

The Tucker decomposition, an extension of singular value decomposition for higher-order tensors, is a useful tool in analysis and compression of large-scale scientific data. While it has been studied extensively for static datasets, there are relatively few works addressing the computation of the Tucker factorization of streaming data tensors. In this paper we propose a new streaming Tucker algorithm tailored for scientific data, specifically for the case of a data tensor whose size increases along a single streaming mode that can grow indefinitely, which is typical of time-stepping scientific applications. At any point of this growth, we seek to compute the Tucker decomposition of the data generated thus far, without requiring storing the past tensor slices in memory. Our algorithm accomplishes this by starting with an initial Tucker decomposition and updating its components--the core tensor and factor matrices--with each new tensor slice as it becomes available, while satisfying a user-specified threshold of norm error. We present an implementation within the TuckerMPI software framework, and apply it to synthetic and combustion simulation datasets. By comparing against the standard (batch) decomposition algorithm we show that our streaming algorithm provides significant improvements in memory usage. If the tensor rank stops growing along the streaming mode, the streaming algorithm also incurs less computational time compared to the batch algorithm.

Jeremy M. Myers, Eric T. Phipps

The Canonical Polyadic (CP) tensor decomposition is a well-known method for interpretable analysis of high-dimensional data. Recently, the Generalized CP method (GCP) was introduced by Hong and Kolda to allow for flexible choice of the loss function in the optimization problem defining the CP model, enabling more interpretable decompositions of strongly non-Gaussian data such as count or binary data. Furthermore, Kolda and Hong introduced a version of GCP that leverages randomization and stochastic optimization to address scalability to large, sparse data sets. In this work, we take these ideas a step further and consider synchronous and asynchronous algorithms for parallel GCP tensor decomposition through the GenTen software package, exploiting both shared and distributed memory parallelism. We build on shared memory parallel CP decomposition algorithms utilizing Kokkos for portability across CPU and GPU architectures to support the random sampling and stochastic optimization methods required by GCP. We then couple this approach to the well-known medium-grained distributed memory parallelism scheme developed for traditional CP decompositions through MPI, providing a synchronous, hybrid MPI+Kokkos, parallel GCP decomposition capability. Finally, we propose an asynchronous distributed parallelism approach building on related techniques for federated learning to achieve even better scalability to large data sets. We study the effectiveness of the proposed synchronous and asynchronous approaches vis-a-vis computational cost and accuracy on synthetic and publicly-available real-world datasets of varying sizes, dimensions, and sparsity patterns using several loss functions.

Matthias Mayr, Alexander Heinlein, Christian Glusa et al.

Trilinos is a community-developed, open-source software framework that facilitates building large-scale, complex, multiscale, multiphysics simulation code bases for scientific and engineering problems. Since the Trilinos framework has undergone substantial changes to support new applications and new hardware architectures, this document is an update to ``An Overview of the Trilinos project'' by Heroux et al. (ACM Transactions on Mathematical Software, 31(3):397-423, 2005). It describes the design of Trilinos, introduces its new organization in product areas, and highlights established and new features available in Trilinos. Particular focus is put on the modernized software stack based on the Kokkos ecosystem to deliver performance portability across heterogeneous hardware architectures. This paper also outlines the organization of the Trilinos community and the contribution model to help onboard interested users and contributors.

Eric Phipps, Nick Johnson, Tamara G. Kolda

In this paper, we develop a method which we call OnlineGCP for computing the Generalized Canonical Polyadic (GCP) tensor decomposition of streaming data. GCP differs from traditional canonical polyadic (CP) tensor decompositions as it allows for arbitrary objective functions which the CP model attempts to minimize. This approach can provide better fits and more interpretable models when the observed tensor data is strongly non-Gaussian. In the streaming case, tensor data is gradually observed over time and the algorithm must incrementally update a GCP factorization with limited access to prior data. In this work, we extend the GCP formalism to the streaming context by deriving a GCP optimization problem to be solved as new tensor data is observed, formulate a tunable history term to balance reconstruction of recently observed data with data observed in the past, develop a scalable solution strategy based on segregated solves using stochastic gradient descent methods, describe a software implementation that provides performance and portability to contemporary CPU and GPU architectures and integrates with Matlab for enhanced useability, and demonstrate the utility and performance of the approach and software on several synthetic and real tensor data sets.

Ramakrishna Tipireddy, Eric T. Phipps, Roger G. Ghanem

In this work, solvers and preconditioners based on Gauss-Seidel and Jacobi algorithms are explored for stochastic Galerkin discretization of partial differential equations (PDEs) with random input data. Gauss-Seidel and Jacobi algorithms are formulated such that the existing software is leveraged in the computational effort. These algorithms are also used as preconditioners to Krylov iterative methods. The solvers and preconditiners are compared with Krylov based iterative methods with the traditional mean-based preconditioner [13] and Kronecker-product preconditioner [17] by solving a steady state state advection-diffusion equation, which upon discretization, results in a non-symmetric positive definite matrix on left-hand-side. Numerical results show that an approximate version of Gauss-Seidel algorithm is a good preconditioner for GMRES to solve non-symmetric Galerkin system of equations.