Fast convergence of the Expectation Maximization algorithm under a logarithmic Sobolev inequality
This work provides a theoretical foundation for understanding and improving EM algorithms, which is incremental as it extends existing techniques to a broader context.
The authors tackled the problem of analyzing the convergence of the Expectation Maximization (EM) algorithm by developing a new framework based on gradient flows and logarithmic Sobolev inequalities, resulting in finite sample error bounds and exponential convergence rates.
We present a new framework for analysing the Expectation Maximization (EM) algorithm. Drawing on recent advances in the theory of gradient flows over Euclidean-Wasserstein spaces, we extend techniques from alternating minimization in Euclidean spaces to the EM algorithm, via its representation as coordinate-wise minimization of the free energy. In so doing, we obtain finite sample error bounds and exponential convergence of the EM algorithm under a natural generalisation of the log-Sobolev inequality. We further show that this framework naturally extends to several variants of EM, offering a unified approach for studying such algorithms.