Bart Vandereycken

Christian Lubich, Ivan Oseledets, Bart Vandereycken

A robust and efficient time integrator for dynamical tensor approximation in the tensor train or matrix product state format is presented. The method is based on splitting the projector onto the tangent space of the tensor manifold. The algorithm can be used for updating time-dependent tensors in the given data-sparse tensor train / matrix product state format and for computing an approximate solution to high-dimensional tensor differential equations within this data-sparse format. The formulation, implementation and theoretical properties of the proposed integrator are studied, and numerical experiments with problems from quantum molecular dynamics and with iterative processes in the tensor train format are included.

Bart Vandereycken

The matrix completion problem consists of finding or approximating a low-rank matrix based on a few samples of this matrix. We propose a new algorithm for matrix completion that minimizes the least-square distance on the sampling set over the Riemannian manifold of fixed-rank matrices. The algorithm is an adaptation of classical non-linear conjugate gradients, developed within the framework of retraction-based optimization on manifolds. We describe all the necessary objects from differential geometry necessary to perform optimization over this low-rank matrix manifold, seen as a submanifold embedded in the space of matrices. In particular, we describe how metric projection can be used as retraction and how vector transport lets us obtain the conjugate search directions. Finally, we prove convergence of a regularized version of our algorithm under the assumption that the restricted isometry property holds for incoherent matrices throughout the iterations. The numerical experiments indicate that our approach scales very well for large-scale problems and compares favorably with the state-of-the-art, while outperforming most existing solvers.

Pieter Lietaert, Javier Pérez, Bart Vandereycken et al.

We present a method for solving nonlinear eigenvalue problems using rational approximation. The method uses the AAA method by Nakatsukasa, Sète, and Trefethen to approximate the nonlinear eigenvalue problem by a rational eigenvalue problem and is embedded in the state space representation of a rational polynomial by Su and Bai. The advantage of the method, compared to related techniques such as NLEIGS and infinite Arnoldi, is the efficient computation by an automatic procedure. In addition, a set-valued approach is developed that allows building a low degree rational approximation of a nonlinear eigenvalue problem. The method perfectly fits the framework of the Compact rational Krylov methods (CORK and TS-CORK), allowing to efficiently solve large scale nonlinear eigenvalue problems. Numerical examples show that the presented framework is competitive with NLEIGS and usually produces smaller linearizations with the same accuracy but with less effort for the user.

Bart Vandereycken, Rik Voorhaar

This work proposes a novel general-purpose estimator for supervised machine learning (ML) based on tensor trains (TT). The estimator uses TTs to parametrize discretized functions, which are then optimized using Riemannian gradient descent under the form of a tensor completion problem. Since this optimization is sensitive to initialization, it turns out that the use of other ML estimators for initialization is crucial. This results in a competitive, fast ML estimator with lower memory usage than many other ML estimators, like the ones used for the initialization.

Marco Sutti, Bart Vandereycken

We propose two implicit numerical schemes for the low-rank time integration of stiff nonlinear partial differential equations. Our approach uses the preconditioned Riemannian trust-region method of Absil, Baker, and Gallivan, 2007. We demonstrate the efficiency of our method for solving the Allen-Cahn and the Fisher-KPP equation on the manifold of fixed-rank matrices. Furthermore, our approach allows us to avoid the restriction on the time step typical of methods that use the fixed-point iteration to solve the inner nonlinear equations. Finally, we demonstrate the efficiency of the preconditioner on the same variational problems presented in Sutti and Vandereycken, 2021.

Marco Sutti, Bart Vandereycken

Large-scale optimization problems arising from the discretization of problems involving PDEs sometimes admit solutions that can be well approximated by low-rank matrices. In this paper, we will exploit this low-rank approximation property by solving the optimization problem directly over the set of low-rank matrices. In particular, we introduce a new multilevel algorithm, where the optimization variable is constrained to the Riemannian manifold of fixed-rank matrices. In contrast to most other multilevel algorithms where the rank is chosen adaptively on each level in order to control the perturbation due to the low-rank truncation, we can keep the ranks (and thus the computational complexity) fixed throughout the iterations. Furthermore, classical implementations of line searches based on Wolfe conditions enable computing a solution where the numerical accuracy is limited to about the square root of the machine epsilon. Here, we propose an extension to Riemannian manifolds of the line search of Hager and Zhang, which uses approximate Wolfe conditions that enable computing a solution on the order of the machine epsilon. Numerical experiments demonstrate the computational efficiency of the proposed framework.

Christian Lubich, Bart Vandereycken, Hanna Walach

Dynamical low-rank approximation in the Tucker tensor format of given large time-dependent tensors and of tensor differential equations is the subject of this paper. In particular, a discrete time integration method for rank-constrained Tucker tensors is presented and analyzed. It extends the known projector-splitting integrator for dynamical low-rank approximation of matrices to Tucker tensors and is shown to inherit the same favorable properties. The integrator is based on iteratively applying the matrix projector-splitting integrator to tensor unfoldings but with inexact solution in a substep. It has the property that it reconstructs time-dependent Tucker tensors of the given rank exactly. The integrator is also shown to be robust to the presence of small singular values in the tensor unfoldings. Numerical examples with problems from quantum dynamics and tensor optimization methods illustrate our theoretical results.

Daniel Kressner, Michael Steinlechner, Bart Vandereycken

The numerical solution of partial differential equations on high-dimensional domains gives rise to computationally challenging linear systems. When using standard discretization techniques, the size of the linear system grows exponentially with the number of dimensions, making the use of classic iterative solvers infeasible. During the last few years, low-rank tensor approaches have been developed that allow to mitigate this curse of dimensionality by exploiting the underlying structure of the linear operator. In this work, we focus on tensors represented in the Tucker and tensor train formats. We propose two preconditioned gradient methods on the corresponding low-rank tensor manifolds: A Riemannian version of the preconditioned Richardson method as well as an approximate Newton scheme based on the Riemannian Hessian. For the latter, considerable attention is given to the efficient solution of the resulting Newton equation. In numerical experiments, we compare the efficiency of our Riemannian algorithms with other established tensor-based approaches such as a truncated preconditioned Richardson method and the alternating linear scheme. The results show that our approximate Riemannian Newton scheme is significantly faster in cases when the application of the linear operator is expensive.