The AL$\ell_0$CORE Tensor Decomposition for Sparse Count Data
This addresses computational efficiency issues for researchers and practitioners working with sparse count data in machine learning and data analysis, representing an incremental improvement over existing tensor decomposition methods.
The paper tackles the problem of high computational cost in Tucker tensor decomposition for sparse count data by introducing ALℓ0CORE, a method that constrains the number of non-zero elements in the core tensor to a preset value Q, achieving results comparable to full Tucker decomposition with only a tiny fraction (e.g., 1%) of the cost.
This paper introduces AL$\ell_0$CORE, a new form of probabilistic non-negative tensor decomposition. AL$\ell_0$CORE is a Tucker decomposition where the number of non-zero elements (i.e., the $\ell_0$-norm) of the core tensor is constrained to a preset value $Q$ much smaller than the size of the core. While the user dictates the total budget $Q$, the locations and values of the non-zero elements are latent variables and allocated across the core tensor during inference. AL$\ell_0$CORE -- i.e., $allo$cated $\ell_0$-$co$nstrained $core$-- thus enjoys both the computational tractability of CP decomposition and the qualitatively appealing latent structure of Tucker. In a suite of real-data experiments, we demonstrate that AL$\ell_0$CORE typically requires only tiny fractions (e.g.,~1%) of the full core to achieve the same results as full Tucker decomposition at only a correspondingly tiny fraction of the cost.