Kullback-Leibler Principal Component for Tensors is not NP-hard
This work addresses computational efficiency in tensor approximation for machine learning and data analysis, providing a practical solution with broad applicability, though it is incremental in extending matrix results to tensors.
The paper tackles the problem of nonnegative rank-one approximation of tensors using generalized Kullback-Leibler divergence, showing that the globally optimal solution is efficiently computable and not NP-hard, with results applicable to arbitrary nonnegative tensors including matrices. It also demonstrates on the Iris dataset that the method achieves performance close to supervised methods in an unsupervised manner.
We study the problem of nonnegative rank-one approximation of a nonnegative tensor, and show that the globally optimal solution that minimizes the generalized Kullback-Leibler divergence can be efficiently obtained, i.e., it is not NP-hard. This result works for arbitrary nonnegative tensors with an arbitrary number of modes (including two, i.e., matrices). We derive a closed-form expression for the KL principal component, which is easy to compute and has an intuitive probabilistic interpretation. For generalized KL approximation with higher ranks, the problem is for the first time shown to be equivalent to multinomial latent variable modeling, and an iterative algorithm is derived that resembles the expectation-maximization algorithm. On the Iris dataset, we showcase how the derived results help us learn the model in an \emph{unsupervised} manner, and obtain strikingly close performance to that from supervised methods.