Singularity structures and impacts on parameter estimation in finite mixtures of distributions
This work addresses the challenge of slow convergence in parameter estimation for statisticians and machine learning practitioners, but it is incremental as it builds on existing theory to analyze specific mixture models.
The authors tackled the problem of parameter estimation in finite mixture models by analyzing singularity structures and their impact on convergence rates, establishing explicit links between singularities, minimax lower bounds, and algebraic geometry, and applied this to skew-normal mixtures to reveal complex asymptotic behaviors not previously reported.
Singularities of a statistical model are the elements of the model's parameter space which make the corresponding Fisher information matrix degenerate. These are the points for which estimation techniques such as the maximum likelihood estimator and standard Bayesian procedures do not admit the root-$n$ parametric rate of convergence. We propose a general framework for the identification of singularity structures of the parameter space of finite mixtures, and study the impacts of the singularity structures on minimax lower bounds and rates of convergence for the maximum likelihood estimator over a compact parameter space. Our study makes explicit the deep links between model singularities, parameter estimation convergence rates and minimax lower bounds, and the algebraic geometry of the parameter space for mixtures of continuous distributions. The theory is applied to establish concrete convergence rates of parameter estimation for finite mixture of skew-normal distributions. This rich and increasingly popular mixture model is shown to exhibit a remarkably complex range of asymptotic behaviors which have not been hitherto reported in the literature.