From Adaptive Kernel Density Estimation to Sparse Mixture Models
This work addresses model complexity and sparsity in mixture modeling for statistical inference, though it appears incremental by building on existing regularization strategies.
The paper tackles the problem of estimating Gaussian mixture models with limited data by introducing a balloon estimator within a generalized expectation-maximization method, resulting in sparse models that reduce the number of effective components as a smoothing parameter increases, bridging from adaptive kernel density estimation to parametric methods.
We introduce a balloon estimator in a generalized expectation-maximization method for estimating all parameters of a Gaussian mixture model given one data sample per mixture component. Instead of limiting explicitly the model size, this regularization strategy yields low-complexity sparse models where the number of effective mixture components reduces with an increase of a smoothing probability parameter $\mathbf{P>0}$. This semi-parametric method bridges from non-parametric adaptive kernel density estimation (KDE) to parametric ordinary least-squares when $\mathbf{P=1}$. Experiments show that simpler sparse mixture models retain the level of details present in the adaptive KDE solution.