Estimating Joint Probability Distribution With Low-Rank Tensor Decomposition, Radon Transforms and Dictionaries
This work addresses the challenge of reliable joint density estimation for data analysis, but it is incremental as it builds on prior methods by integrating existing ideas.
The paper tackles the problem of estimating joint probability distributions from data samples by proposing a method that combines low-rank tensor decomposition with dictionaries and Radon transforms, resulting in improved sample complexity and outperforming previous dictionary-based approaches and Gaussian Mixture Models in all experimental settings.
In this paper, we describe a method for estimating the joint probability density from data samples by assuming that the underlying distribution can be decomposed as a mixture of product densities with few mixture components. Prior works have used such a decomposition to estimate the joint density from lower-dimensional marginals, which can be estimated more reliably with the same number of samples. We combine two key ideas: dictionaries to represent 1-D densities, and random projections to estimate the joint distribution from 1-D marginals, explored separately in prior work. Our algorithm benefits from improved sample complexity over the previous dictionary-based approach by using 1-D marginals for reconstruction. We evaluate the performance of our method on estimating synthetic probability densities and compare it with the previous dictionary-based approach and Gaussian Mixture Models (GMMs). Our algorithm outperforms these other approaches in all the experimental settings.