A Riemannian Newton Trust-Region Method for Fitting Gaussian Mixture Models
This work addresses a bottleneck in clustering and density approximation for data science and statistics, offering a more efficient optimization method for GMMs, though it is incremental as it builds on existing manifold optimization techniques.
The authors tackled the slow convergence of Expectation Maximization (EM) for Gaussian Mixture Models (GMMs) in cases with hidden information or overlapping clusters by introducing a Riemannian Newton Trust-Region method with an explicit Riemannian Hessian formula, resulting in improved runtime and iteration counts compared to current approaches.
Gaussian Mixture Models are a powerful tool in Data Science and Statistics that are mainly used for clustering and density approximation. The task of estimating the model parameters is in practice often solved by the Expectation Maximization (EM) algorithm which has its benefits in its simplicity and low per-iteration costs. However, the EM converges slowly if there is a large share of hidden information or overlapping clusters. Recent advances in manifold optimization for Gaussian Mixture Models have gained increasing interest. We introduce an explicit formula for the Riemannian Hessian for Gaussian Mixture Models. On top, we propose a new Riemannian Newton Trust-Region method which outperforms current approaches both in terms of runtime and number of iterations. We apply our method on clustering problems and density approximation tasks. Our method is very powerful for data with a large share of hidden information compared to existing methods.