Block-Randomized Stochastic Proximal Gradient for Low-Rank Tensor Factorization
This work addresses a bottleneck in tensor factorization for applications like medical imaging and computer vision by enabling efficient handling of dense tensors with constraints, though it is incremental as it builds on existing stochastic methods.
The paper tackles the problem of computing canonical polyadic decomposition for large dense tensors by proposing a stochastic optimization framework that combines randomized block coordinate descent and stochastic proximal gradient, achieving lightweight updates and flexibility with constraints/regularizations, supported by convergence analysis and numerical results on large-scale dense tensors.
This work considers the problem of computing the canonical polyadic decomposition (CPD) of large tensors. Prior works mostly leverage data sparsity to handle this problem, which is not suitable for handling dense tensors that often arise in applications such as medical imaging, computer vision, and remote sensing. Stochastic optimization is known for its low memory cost and per-iteration complexity when handling dense data. However, exisiting stochastic CPD algorithms are not flexible enough to incorporate a variety of constraints/regularizations that are of interest in signal and data analytics. Convergence properties of many such algorithms are also unclear. In this work, we propose a stochastic optimization framework for large-scale CPD with constraints/regularizations. The framework works under a doubly randomized fashion, and can be regarded as a judicious combination of randomized block coordinate descent (BCD) and stochastic proximal gradient (SPG). The algorithm enjoys lightweight updates and small memory footprint. In addition, this framework entails considerable flexibility---many frequently used regularizers and constraints can be readily handled under the proposed scheme. The approach is also supported by convergence analysis. Numerical results on large-scale dense tensors are employed to showcase the effectiveness of the proposed approach.