MM for Penalized Estimation
This work addresses the need for reliable variable selection and parameter estimation in statistical modeling when data are contaminated with outliers, offering an incremental improvement by extending MM algorithms to nonconvex loss functions.
The paper tackles the problem of robust penalized estimation for data with outliers by developing a majorization-minimization (MM) algorithm that works with both convex and nonconvex loss functions, providing optimality conditions and convergence theory, and demonstrates performance on simulated and real datasets like healthcare costs and cancer clinical status.
Penalized estimation can conduct variable selection and parameter estimation simultaneously. The general framework is to minimize a loss function subject to a penalty designed to generate sparse variable selection. The majorization-minimization (MM) algorithm is a computational scheme for stability and simplicity, and the MM algorithm has been widely applied in penalized estimation. Much of the previous work have focused on convex loss functions such as generalized linear models. When data are contaminated with outliers, robust loss functions can generate more reliable estimates. Recent literature has witnessed a growing impact of nonconvex loss-based methods, which can generate robust estimation for data contaminated with outliers. This article investigates MM algorithm for penalized estimation, provide innovative optimality conditions and establish convergence theory with both convex and nonconvex loss functions. With respect to applications, we focus on several nonconvex loss functions, which were formerly studied in machine learning for regression and classification problems. Performance of the proposed algorithms are evaluated on simulated and real data including healthcare costs and cancer clinical status. Efficient implementations of the algorithms are available in the R package mpath in CRAN.