Consistent Density Estimation Under Discrete Mixture Models
This work addresses density estimation for statisticians and machine learning practitioners, providing theoretical guarantees but is incremental as it builds on existing mixture model frameworks.
The paper tackles the problem of estimating a mixing probability density in discrete mixture models, proving the existence of an L1 consistent estimator under bijective mapping conditions and demonstrating computational feasibility, with a case study in Poisson mixture models showing consistency.
This work considers a problem of estimating a mixing probability density $f$ in the setting of discrete mixture models. The paper consists of three parts. The first part focuses on the construction of an $L_1$ consistent estimator of $f$. In particular, under the assumptions that the probability measure $μ$ of the observation is atomic, and the map from $f$ to $μ$ is bijective, it is shown that there exists an estimator $f_n$ such that for every density $f$ $\lim_{n\to \infty} \mathbb{E} \left[ \int |f_n -f | \right]=0$. The second part discusses the implementation details. Specifically, it is shown that the consistency for every $f$ can be attained with a computationally feasible estimator. The third part, as a study case, considers a Poisson mixture model. In particular, it is shown that in the Poisson noise setting, the bijection condition holds and, hence, estimation can be performed consistently for every $f$.