Shmuel Friedland

Shmuel Friedland, Giorgio Ottaviani

In this paper we discuss the notion of singular vector tuples of a complex valued $d$-mode tensor of dimension m_1 x ... x m_d. We show that a generic tensor has a finite number of singular vector tuples, viewed as points in the corresponding Segre product. We give the formula for the number of singular vector tuples. We show similar results for tensors with partial symmetry. We give analogous results for the homogeneous pencil eigenvalue problem for cubic tensors, i.e. m_1=...=m_d. We show uniqueness of best approximations for almost all real tensors in the following cases: rank one approximation; rank one approximation for partially symmetric tensors (this approximation is also partially symmetric); rank-(r_1,...,r_d) approximation for $d$-mode tensors.

Shmuel Friedland

We study the generic and typical ranks of 3-tensors of dimension l x m x n using results from matrices and algebraic geometry. We state a conjecture about the exact values of the generic rank of 3-tensors over the complex numbers, which is verified numerically for l,m,n not greater than 14. We also discuss the typical ranks over the real numbers, and give an example of an infinite family of 3-tensors of the form l=m, n=(m-1)^2+1, m=3,4,..., which have at least two typical ranks.

Shmuel Friedland, Chee Wei Tan

We study the problem of maximizing sum rates in a Gaussian interference-limited channel that models multiuser communication in a CDMA wireless network or DSL cable binder. Using tools from nonnegative irreducible matrix theory, in particular the Perron-Frobenius Theorem and the Friedland-Karlin inequalities, we provide insights into the structural property of optimal power allocation strategies that maximize sum rates. Our approach is similar to the treatment of linear models in mathematical economies, where interference is viewed in the context of competition. We show that this maximum problem can be restated as a maximization problem of a convex function on a closed convex set. We suggest three algorithms to find the exact and approximate values of the optimal sum rates. In particular, our algorithms exploit the eigenspace of specially crafted nonnegative {\it interference matrices}, which, with the use of standard optimization tools, can provide useful upper bounds and feasible solutions to the nonconvex problem.

Shmuel Friedland

Assume that f is a strict convex function with a unique minimum in R^n. We divide the vector of n-variables to d groups of vector subvariables with d at least two. We assume that we can find the partial minimum of f with respect to each vector subvariable while other variables are fixed. We then describe an algorithm that partially minimizes each time on a specifically chosen vector subvariable. This algorithm converges geometrically to the unique minimum. The rate of convergence depends on the uniform bounds on the eigenvalues of the Hessian of f in the compact sublevel set f whose values are at most f(x_0), where x_0 is the starting point of the algorithm. In the case where f is a polynomial of degree two, with positive definite quadratic term, and d=n our method can be considered as a generalization of the classical conjugate gradient method. The main result of this paper is the observation that the celebrated Sinkhorn diagonal scaling algorithm for matrices, and the corresponding diagonal scaling of tensors, can be viewed as partial minimization of certain logconvex functions.

Shmuel Friedland

We study the optimal transport problem for $d>2$ discrete measures. This is a linear programming problem on $d$-tensors. It gives a way to compute a "distance" between two sets of discrete measures. We introduce an entropic regularization term, which gives rise to a scaling of tensors. We give a variation of the celebrated Sinkhorn scaling algorithm. We show that this algorithm can be viewed as a partial minimization algorithm of a strictly convex function. Under appropriate conditions the rate of convergence is geometric and we estimate the rate. Our results are generalizations of known results for the classical case of two discrete measures.

Alwin Stegeman, Shmuel Friedland

It is well known that a best rank-$R$ approximation of order-3 tensors may not exist for $R\ge 2$. A best rank-$(R,R,R)$ approximation always exists, however, and is also a best rank-$R$ approximation when it has rank (at most) $R$. For $R=2$ and real order-3 tensors it is shown that a best rank-2 approximation is also a local minimum of the best rank-(2,2,2) approximation problem. This implies that if all rank-(2,2,2) minima have rank larger than 2, then a best rank-2 approximation does not exist. This provides an easy-to-check criterion for existence of a best rank-2 approximation. The result is illustrated by means of simulations.

Shmuel Friedland, Venu Tammali

In many applications such as data compression, imaging or genomic data analysis, it is important to approximate a given tensor by a tensor that is sparsely representable. For matrices, i.e. 2-tensors, such a representation can be obtained via the singular value decomposition, which allows to compute best rank k-approximations. For very big matrices a low rank approximation using SVD is not computationally feasible. In this case different approximations are available. It seems that variants of CUR decomposition are most suitable. For d-mode tensors T with d>2, many generalizations of the singular value decomposition have been proposed to obtain low tensor rank decompositions. The most appropriate approximation seems to be best (r_1,...,r_d)-approximation, which maximizes the l_2 norm of the projection of T on a tensor product of subspaces U_1,...,U_d, where U_i is an r_i-dimensional subspace. One of the most common method is the alternating maximization method (AMM). It is obtained by maximizing on one subspace U_i, while keeping all other fixed, and alternating the procedure repeatedly for i=1,...,d. Usually, AMM will converge to a local best approximation. This approximation is a fixed point of a corresponding map on Grassmannians. We suggest a Newton method for finding the corresponding fixed point. We also discuss variants of CUR-approximation method for tensors. The first part of the paper is a survey on low rank approximation of tensors. The second new part of this paper is a new Newton method for best $(r_1,...,r_d)$-approximation. We compare numerically different approximation methods.

Shmuel Friedland, Qun Li, Dan Schonfeld

Compressive sensing (CS) has triggered enormous research activity since its first appearance. CS exploits the signal's sparsity or compressibility in a particular domain and integrates data compression and acquisition, thus allowing exact reconstruction through relatively few non-adaptive linear measurements. While conventional CS theory relies on data representation in the form of vectors, many data types in various applications such as color imaging, video sequences, and multi-sensor networks, are intrinsically represented by higher-order tensors. Application of CS to higher-order data representation is typically performed by conversion of the data to very long vectors that must be measured using very large sampling matrices, thus imposing a huge computational and memory burden. In this paper, we propose Generalized Tensor Compressive Sensing (GTCS)--a unified framework for compressive sensing of higher-order tensors which preserves the intrinsic structure of tensor data with reduced computational complexity at reconstruction. GTCS offers an efficient means for representation of multidimensional data by providing simultaneous acquisition and compression from all tensor modes. In addition, we propound two reconstruction procedures, a serial method (GTCS-S) and a parallelizable method (GTCS-P). We then compare the performance of the proposed method with Kronecker compressive sensing (KCS) and multi way compressive sensing (MWCS). We demonstrate experimentally that GTCS outperforms KCS and MWCS in terms of both reconstruction accuracy (within a range of compression ratios) and processing speed. The major disadvantage of our methods (and of MWCS as well), is that the compression ratios may be worse than that offered by KCS.

Shmuel Friedland

In this paper we give necessary and sufficient conditions on a nonnegative tensor to be diagonally equivalent to a tensor with prescribed slice sums. These conditions are variations of Bapat-Raghavan and Franklin-Lorenz conditions.

Shmuel Friedland, Per Hakan Lundow, Klas Markstrom

The notion of a 1-vertex transfer matrix for multi-dimensional codes is introduced. It is shown that the capacity of such codes, or the topological entropy, can be expressed as the limit of the logarithm of spectral radii of 1-vertex transfer matrices. Storage and computations using the 1-vertex transfer matrix are much smaller than storage and computations needed for the standard transfer matrix. The method is applied to estimate the first 15 digits of the entropy of the 2-dimensional (0,1) run length limited channel. In order to compare the computational cost of the new method with the standard transfer matrix and have rigorous bounds to compare the estimates with a large scale computation of eigenvalues for the (0,1) run length limited channel in 2 and 3 dimensions have been carried out. This in turn leads to improvements on the best previous lower and upper bounds for that channel.

Shmuel Friedland, Anatoli Torokhti

In this paper we give an explicit solution to the rank constrained matrix approximation in Frobenius norm, which is a generalization of the classical approximation of an m by n matrix A by a matrix of rank k at most.

Shmuel Friedland, Mostafa Kaveh, Amir Niknejad et al.

In many applications, it is of interest to approximate data, given by mxn matrix A, by a matrix B of at most rank k, which is much smaller than m and n. The best approximation is given by singular value decomposition, which is too time consuming for very large m and n. We present here a Monte Carlo algorithm for iteratively computing a k-rank approximation to the data consisting of mxn matrix A. Each iteration involves the reading of O(k) of columns or rows of A. The complexity of our algorithm is O(kmn). Our algorithm, distinguished from other known algorithms, guarantees that each iteration is a better k-rank approximation than the previous iteration. We believe that this algorithm will have many applications in data mining, data storage and data analysis.

Shmuel Friedland, Mostafa Kaveh, Amir Niknejad et al.

Gene expression data matrices often contain missing expression values. In this paper, we describe a new algorithm, named improved fixed rank approximation algorithm (IFRAA), for missing values estimations of the large gene expression data matrices. We compare the present algorithm with the two existing and widely used methods for reconstructing missing entries for DNA microarray gene expression data: the Bayesian principal component analysis (BPCA) and the local least squares imputation method (LLS). The three algorithms were applied to four microarray data sets and two synthetic low-rank data matrices. Certain percentages of the elements of these data sets were randomly deleted, and the three algorithms were used to recover them. In conclusion IFRAA appears to be the most reliable and accurate approach for recovering missing DNA microarray gene expression data, or any other noisy data matrices that are effectively low rank.