Learning Mixtures of Smooth Product Distributions: Identifiability and Algorithm
This addresses the challenge of learning mixture models in a non-parametric setting, which is incremental as it extends existing parametric methods to more flexible distributions.
The paper tackles the problem of learning mixtures of non-parametric product distributions by proposing a two-stage approach that recovers component distributions under smoothness conditions, demonstrating effectiveness on synthetic and real datasets.
We study the problem of learning a mixture model of non-parametric product distributions. The problem of learning a mixture model is that of finding the component distributions along with the mixing weights using observed samples generated from the mixture. The problem is well-studied in the parametric setting, i.e., when the component distributions are members of a parametric family -- such as Gaussian distributions. In this work, we focus on multivariate mixtures of non-parametric product distributions and propose a two-stage approach which recovers the component distributions of the mixture under a smoothness condition. Our approach builds upon the identifiability properties of the canonical polyadic (low-rank) decomposition of tensors, in tandem with Fourier and Shannon-Nyquist sampling staples from signal processing. We demonstrate the effectiveness of the approach on synthetic and real datasets.