Density estimation via mixture discrepancy and moments
This work addresses computational bottlenecks in density estimation for researchers in statistics and machine learning, but it is incremental as it builds directly on prior methods.
The paper tackled the computational inefficiency and lack of invariance in density estimation using star discrepancy by proposing two new methods, DSP-mix and MSP, which replace star discrepancy with mixture discrepancy and moments, respectively. The result showed that both new methods run about ten times faster than the previous DSP method while maintaining the same accuracy in reconstructing multi-dimensional Gaussian and Beta mixtures.
With the aim of generalizing histogram statistics to higher dimensional cases, density estimation via discrepancy based sequential partition (DSP) has been proposed [D. Li, K. Yang, W. Wong, Advances in Neural Information Processing Systems (2016) 1099-1107] to learn an adaptive piecewise constant approximation defined on a binary sequential partition of the underlying domain, where the star discrepancy is adopted to measure the uniformity of particle distribution. However, the calculation of the star discrepancy is NP-hard and it does not satisfy the reflection invariance and rotation invariance either. To this end, we use the mixture discrepancy and the comparison of moments as a replacement of the star discrepancy, leading to the density estimation via mixture discrepancy based sequential partition (DSP-mix) and density estimation via moments based sequential partition (MSP), respectively. Both DSP-mix and MSP are computationally tractable and exhibit the reflection and rotation invariance. Numerical experiments in reconstructing the $d$-D mixture of Gaussians and Betas with $d=2, 3, \dots, 6$ demonstrate that DSP-mix and MSP both run approximately ten times faster than DSP while maintaining the same accuracy.