Near-Universal Multiplicative Updates for Nonnegative Einsum Factorization
This provides a practical solution for researchers across scientific domains dealing with multiway data, enabling easy customization and efficient computation, though it is incremental in improving existing factorization methods.
The paper tackles the lack of user-friendly tools for nonnegative tensor factorization by introducing NNEinFact, an einsum-based multiplicative update algorithm that fits custom models, achieving over 37% better performance in heldout prediction tasks and up to 90 times faster convergence compared to gradient-based methods.
Despite the ubiquity of multiway data across scientific domains, there are few user-friendly tools that fit tailored nonnegative tensor factorizations. Researchers may use gradient-based automatic differentiation (which often struggles in nonnegative settings), choose between a limited set of methods with mature implementations, or implement their own model from scratch. As an alternative, we introduce NNEinFact, an einsum-based multiplicative update algorithm that fits any nonnegative tensor factorization expressible as a tensor contraction by minimizing one of many user-specified loss functions (including the $(α,β)$-divergence). To use NNEinFact, the researcher simply specifies their model with a string. NNEinFact converges to a local minimum of the loss, supports missing data, and fits to tensors with hundreds of millions of entries in seconds. Empirically, NNEinFact fits custom models which outperform standard ones in heldout prediction tasks on real-world tensor data by over $37\%$ and attains less than half the test loss of gradient-based methods while converging up to 90 times faster.