Noise Benefits in Expectation-Maximization Algorithms
It addresses the problem of slow convergence in iterative algorithms for statisticians and machine learning practitioners, offering a novel approach to improve efficiency.
This dissertation demonstrates that injecting noise into sample data can significantly accelerate Expectation-Maximization algorithms, a class used for maximum likelihood estimation from incomplete data, by showing speed-ups in algorithms like k-means and backpropagation.
This dissertation shows that careful injection of noise into sample data can substantially speed up Expectation-Maximization algorithms. Expectation-Maximization algorithms are a class of iterative algorithms for extracting maximum likelihood estimates from corrupted or incomplete data. The convergence speed-up is an example of a noise benefit or "stochastic resonance" in statistical signal processing. The dissertation presents derivations of sufficient conditions for such noise-benefits and demonstrates the speed-up in some ubiquitous signal-processing algorithms. These algorithms include parameter estimation for mixture models, the $k$-means clustering algorithm, the Baum-Welch algorithm for training hidden Markov models, and backpropagation for training feedforward artificial neural networks. This dissertation also analyses the effects of data and model corruption on the more general Bayesian inference estimation framework. The main finding is a theorem guaranteeing that uniform approximators for Bayesian model functions produce uniform approximators for the posterior pdf via Bayes theorem. This result also applies to hierarchical and multidimensional Bayesian models.