Completing a joint PMF from projections: a low-rank coupled tensor factorization approach
This addresses the challenge of statistical estimation in machine learning by enabling reconstruction of joint distributions from limited data, though it is incremental as it builds on existing low-rank matrix and tensor completion methods.
The paper tackles the problem of estimating high-dimensional joint probability distributions from lower-dimensional marginal projections by proposing a coupled low-rank tensor factorization approach, which guarantees identifiability under low-rank conditions and demonstrates effectiveness in rating prediction.
There has recently been considerable interest in completing a low-rank matrix or tensor given only a small fraction (or few linear combinations) of its entries. Related approaches have found considerable success in the area of recommender systems, under machine learning. From a statistical estimation point of view, the gold standard is to have access to the joint probability distribution of all pertinent random variables, from which any desired optimal estimator can be readily derived. In practice high-dimensional joint distributions are very hard to estimate, and only estimates of low-dimensional projections may be available. We show that it is possible to identify higher-order joint PMFs from lower-order marginalized PMFs using coupled low-rank tensor factorization. Our approach features guaranteed identifiability when the full joint PMF is of low-enough rank, and effective approximation otherwise. We provide an algorithmic approach to compute the sought factors, and illustrate the merits of our approach using rating prediction as an example.