Lars Karlsson

Zvonimir Bujanović, Lars Karlsson, Daniel Kressner

The QZ algorithm for computing eigenvalues and eigenvectors of a matrix pencil $A - λB$ requires that the matrices first be reduced to Hessenberg-triangular (HT) form. The current method of choice for HT reduction relies entirely on Givens rotations regrouped and accumulated into small dense matrices which are subsequently applied using matrix multiplication routines. A non-vanishing fraction of the total flop count must nevertheless still be performed as sequences of overlapping Givens rotations alternately applied from the left and from the right. The many data dependencies associated with this computational pattern leads to inefficient use of the processor and poor scalability. In this paper, we therefore introduce a fundamentally different approach that relies entirely on (large) Householder reflectors partially accumulated into block reflectors, by using (compact) WY representations. Even though the new algorithm requires more floating point operations than the state of the art algorithm, extensive experiments on both real and synthetic data indicate that it is still competitive, even in a sequential setting. The new algorithm is conjectured to have better parallel scalability, an idea which is partially supported by early small-scale experiments using multi-threaded BLAS. The design and evaluation of a parallel formulation is future work.

Per Sehlstedt, Jan Brandejs, Paolo Bientinesi et al.

The density matrix renormalization group (DMRG) algorithm is a cornerstone computational method for studying quantum many-body systems, renowned for its accuracy and adaptability. Despite DMRG's broad applicability across fields such as materials science, quantum chemistry, and quantum computing, numerous independent implementations have been developed. This survey maps the rapidly expanding DMRG software landscape, providing a comprehensive comparison of features among 35 existing packages. We found significant overlap in features among the packages when comparing key aspects, such as parallelism strategies for high-performance computing and symmetry-adapted formulations that enhance efficiency. This overlap suggests opportunities for modularization of common operations, including tensor operations, symmetry representations, and eigensolvers, as the packages are mostly independent and share few third-party library dependencies where functionality is factored out. More widespread modularization and standardization would result in reduced duplication of efforts and improved interoperability. We believe that the proliferation of packages and the current lack of standard interfaces and modularity are more social than technical. We aim to raise awareness of existing packages, guide researchers in finding a suitable package for their needs, and help developers identify opportunities for collaboration, modularity standardization, and optimization. Ultimately, this work emphasizes the value of greater cohesion and modularity, which would benefit DMRG software, allowing these powerful algorithms to tackle more complex and ambitious problems.

Christos Psarras, Lars Karlsson, Rasmus Bro et al.

Tensor decompositions, such as CANDECOMP/PARAFAC (CP), are widely used in a variety of applications, such as chemometrics, signal processing, and machine learning. A broadly used method for computing such decompositions relies on the Alternating Least Squares (ALS) algorithm. When the number of components is small, regardless of its implementation, ALS exhibits low arithmetic intensity, which severely hinders its performance and makes GPU offloading ineffective. We observe that, in practice, experts often have to compute multiple decompositions of the same tensor, each with a small number of components (typically fewer than 20), to ultimately find the best ones to use for the application at hand. In this paper, we illustrate how multiple decompositions of the same tensor can be fused together at the algorithmic level to increase the arithmetic intensity. Therefore, it becomes possible to make efficient use of GPUs for further speedups; at the same time the technique is compatible with many enhancements typically used in ALS, such as line search, extrapolation, and non-negativity constraints. We introduce the Concurrent ALS algorithm and library, which offers an interface to Matlab, and a mechanism to effectively deal with the issue that decompositions complete at different times. Experimental results on artificial and real datasets demonstrate a shorter time to completion due to increased arithmetic intensity.