I. V. Oseledets

A. Yu Mikhalev, I. V. Oseledets

A new fast algebraic method for obtaining an $\mathcal{H}^2$-approximation of a matrix from its entries is presented. The main idea behind the method is based on the nested representation and the maximum-volume principle to select submatrices in low-rank matrices. A special iterative approach for the computation of so-called representing sets is established. The main advantage of the method is that it uses only the hierarchical partitioning of the matrix and does not require special "proxy surfaces" to be selected in advance. The numerical experiments for the electrostatic problem and for the boundary integral operator confirm the effectiveness and robustness of the approach. The complexity is linear in the matrix size and polynomial in the ranks. The algorithm is implemented as an open-source Python package that is available online.

A. V. Chertkov, I. V. Oseledets, M. V. Rakhuba

In this work we propose an efficient black-box solver for two-dimensional stationary diffusion equations, which is based on a new robust discretization scheme. The idea is to formulate an equation in a certain form without derivatives with a non-local stencil, which leads us to a linear system of equations with dense matrix. This matrix and a right-hand side are represented in a low-rank parametric representation -- the quantized tensor train (QTT-) format, and then all operations are performed with logarithmic complexity and memory consumption. Hence very fine grids can be used, and very accurate solutions with extremely high spatial resolution can be obtained. Numerical experiments show that this formulation gives accurate results and can be used up to $2^{60}$ grid points with no problems with conditioning, while total computational time is around several seconds.

M. S. Litsarev, I. V. Oseledets

We present a method for solving the reaction-diffusion equation with general potential in free space. It is based on the approximation of the Feynman-Kac formula by a sequence of convolutions on sequentially diminishing grids. For computation of the convolutions we propose a fast algorithm based on the low-rank approximation of the Hankel matrices. The algorithm has complexity of $\mathcal{O}(nr M \log M + nr^2 M)$ flops and requires $\mathcal{O}(M r)$ floating-point numbers in memory, where $n$ is the dimension of the integral, $r \ll n$, and $M$ is the mesh size in one dimension. The presented technique can be generalized to the higher-order diffusion processes.

V. P. Zankin, G. V. Ryzhakov, I. V. Oseledets

In this work, we propose a novel sampling method for Design of Experiments. This method allows to sample such input values of the parameters of a computational model for which the constructed surrogate model will have the least possible approximation error. High efficiency of the proposed method is demonstrated by its comparison with other sampling techniques (LHS, Sobol' sequence sampling, and Maxvol sampling) on the problem of least-squares polynomial approximation. Also, numerical experiments for the Lebesgue constant growth for the points sampled by the proposed method are carried out.

A. L. Pavlov, G. V. Ovchinnikov, D. Yu. Derbyshev et al.

Iterative Closest Point (ICP) is a widely used method for performing scan-matching and registration. Being simple and robust method, it is still computationally expensive and may be challenging to use in real-time applications with limited resources on mobile platforms. In this paper we propose novel effective method for acceleration of ICP which does not require substantial modifications to the existing code. This method is based on an idea of Anderson acceleration which is an iterative procedure for finding a fixed point of contractive mapping. The latter is often faster than a standard Picard iteration, usually used in ICP implementations. We show that ICP, being a fixed point problem, can be significantly accelerated by this method enhanced by heuristics to improve overall robustness. We implement proposed approach into Point Cloud Library (PCL) and make it available online. Benchmarking on real-world data fully supports our claims.

A. Cichocki, A-H. Phan, Q. Zhao et al.

Part 2 of this monograph builds on the introduction to tensor networks and their operations presented in Part 1. It focuses on tensor network models for super-compressed higher-order representation of data/parameters and related cost functions, while providing an outline of their applications in machine learning and data analytics. A particular emphasis is on the tensor train (TT) and Hierarchical Tucker (HT) decompositions, and their physically meaningful interpretations which reflect the scalability of the tensor network approach. Through a graphical approach, we also elucidate how, by virtue of the underlying low-rank tensor approximations and sophisticated contractions of core tensors, tensor networks have the ability to perform distributed computations on otherwise prohibitively large volumes of data/parameters, thereby alleviating or even eliminating the curse of dimensionality. The usefulness of this concept is illustrated over a number of applied areas, including generalized regression and classification (support tensor machines, canonical correlation analysis, higher order partial least squares), generalized eigenvalue decomposition, Riemannian optimization, and in the optimization of deep neural networks. Part 1 and Part 2 of this work can be used either as stand-alone separate texts, or indeed as a conjoint comprehensive review of the exciting field of low-rank tensor networks and tensor decompositions.

A. Cichocki, N. Lee, I. V. Oseledets et al.

Machine learning and data mining algorithms are becoming increasingly important in analyzing large volume, multi-relational and multi--modal datasets, which are often conveniently represented as multiway arrays or tensors. It is therefore timely and valuable for the multidisciplinary research community to review tensor decompositions and tensor networks as emerging tools for large-scale data analysis and data mining. We provide the mathematical and graphical representations and interpretation of tensor networks, with the main focus on the Tucker and Tensor Train (TT) decompositions and their extensions or generalizations. Keywords: Tensor networks, Function-related tensors, CP decomposition, Tucker models, tensor train (TT) decompositions, matrix product states (MPS), matrix product operators (MPO), basic tensor operations, multiway component analysis, multilinear blind source separation, tensor completion, linear/multilinear dimensionality reduction, large-scale optimization problems, symmetric eigenvalue decomposition (EVD), PCA/SVD, huge systems of linear equations, pseudo-inverse of very large matrices, Lasso and Canonical Correlation Analysis (CCA) (This is Part 1)

G. V. Ovchinnikov, D. Zorin, I. V. Oseledets

Topological optimization finds a material density distribution minimizing a functional of the solution of a partial differential equation (PDE), subject to a set of constraints (typically, a bound on the volume or mass of the material). Using a finite elements discretization (FEM) of the PDE and functional we obtain an integer programming problem. Due to approximation error of FEM discretization, optimization problem becomes mesh-depended and possess false, physically inadequate optimums, while functional value heavily depends on fineness of discretization scheme used to compute it. To alleviate this problem, we propose regularization of given functional by error estimate of FEM discretization. This regularization provides robustness of solutions and improves obtained functional values as well. While the idea is broadly applicable, in this paper we apply our method to the heat conduction optimization. This type of problems are of practical importance in design of heat conduction channels, heat sinks and other types of heat guides.

M. V. Rakhuba, I. V. Oseledets

We propose a new cross-conv algorithm for approximate computation of convolution in different low-rank tensor formats (tensor train, Tucker, Hierarchical Tucker). It has better complexity with respect to the tensor rank than previous approaches. The new algorithm has a high potential impact in different applications. The key idea is based on applying cross approximation in the "frequency domain", where convolution becomes a simple elementwise product. We illustrate efficiency of our algorithm by computing the three-dimensional Newton potential and by presenting preliminary results for solution of the Hartree-Fock equation on tensor-product grids.

D. A. Kolesnikov, I. V. Oseledets

We propose a new method for the approximate solution of the Lyapunov equation with rank-$1$ right-hand side, which is based on extended rational Krylov subspace approximation with adaptively computed shifts. The shift selection is obtained from the connection between the Lyapunov equation, solution of systems of linear ODEs and alternating least squares method for low-rank approximation. The numerical experiments confirm the effectiveness of our approach.

M. S. Litsarev, I. V. Oseledets

An efficient technique based on low-rank separated approximations is proposed for computation of three-dimensional integrals arising in the energy deposition model that describes ion-atomic collisions. Direct tensor-product quadrature requires grids of size $4000^3$ which is unacceptable. Moreover, several of such integrals have to be computed simultaneously for different values of parameters. To reduce the complexity, we use the structure of the integrand and apply numerical linear algebra techniques for the construction of low-rank approximation. The resulting algorithm is $10^3$ faster than spectral quadratures in spherical coordinates used in the original DEPOSIT code. The approach can be generalized to other multidimensional problems in physics.