Coordinate Descent for MCP/SCAD Penalized Least Squares Converges Linearly
This work addresses computational efficiency for sparse signal recovery in statistics and machine learning, but it is incremental as it extends known convergence proofs to specific penalties.
The authors tackled the problem of proving convergence rates for coordinate descent applied to nonconvex penalized least squares, specifically for MCP/SCAD penalties, and established a linear convergence rate as the result.
Recovering sparse signals from observed data is an important topic in signal/imaging processing, statistics and machine learning. Nonconvex penalized least squares have been attracted a lot of attentions since they enjoy nice statistical properties. Computationally, coordinate descent (CD) is a workhorse for minimizing the nonconvex penalized least squares criterion due to its simplicity and scalability. In this work, we prove the linear convergence rate to CD for solving MCP/SCAD penalized least squares problems.