An Introduction to MM Algorithms for Machine Learning and Statistical
It provides a tutorial on an existing optimization method for researchers and practitioners in machine learning and statistics, but is incremental as it does not propose new algorithms or significant improvements.
The paper introduces the MM (majorization-minimization) algorithm framework for solving optimization problems in machine learning and statistical estimation, demonstrating its application through examples like Gaussian mixture regressions and support vector machines with numerical demonstrations.
MM (majorization--minimization) algorithms are an increasingly popular tool for solving optimization problems in machine learning and statistical estimation. This article introduces the MM algorithm framework in general and via three popular example applications: Gaussian mixture regressions, multinomial logistic regressions, and support vector machines. Specific algorithms for the three examples are derived and numerical demonstrations are presented. Theoretical and practical aspects of MM algorithm design are discussed.