Learning Mixtures of Low-Rank Models
This addresses a foundational challenge in machine learning by combining latent variables and structural priors, though it is incremental in extending low-rank matrix sensing and mixed linear regression.
The paper tackles the problem of learning mixtures of low-rank models by reconstructing multiple low-rank matrices from unlabelled linear measurements, achieving near-optimal sample and computational complexities with provable stability against noise.
We study the problem of learning mixtures of low-rank models, i.e. reconstructing multiple low-rank matrices from unlabelled linear measurements of each. This problem enriches two widely studied settings -- low-rank matrix sensing and mixed linear regression -- by bringing latent variables (i.e. unknown labels) and structural priors (i.e. low-rank structures) into consideration. To cope with the non-convexity issues arising from unlabelled heterogeneous data and low-complexity structure, we develop a three-stage meta-algorithm that is guaranteed to recover the unknown matrices with near-optimal sample and computational complexities under Gaussian designs. In addition, the proposed algorithm is provably stable against random noise. We complement the theoretical studies with empirical evidence that confirms the efficacy of our algorithm.