Nonconvex Penalization in Sparse Estimation: An Approach Based on the Bernstein Function
This work addresses sparse estimation problems in statistics and machine learning, presenting an incremental method for nonconvex penalization.
The paper tackles high-dimensional sparse estimation by introducing a class of nonconvex penalty functions based on Bernstein functions, deriving thresholding functions and algorithms like coordinate descent and proximal alternating linearized minimization, with convergence analysis using the Kurdyka-Lojasiewicz inequality.
In this paper we study nonconvex penalization using Bernstein functions whose first-order derivatives are completely monotone. The Bernstein function can induce a class of nonconvex penalty functions for high-dimensional sparse estimation problems. We derive a thresholding function based on the Bernstein penalty and discuss some important mathematical properties in sparsity modeling. We show that a coordinate descent algorithm is especially appropriate for regression problems penalized by the Bernstein function. We also consider the application of the Bernstein penalty in classification problems and devise a proximal alternating linearized minimization method. Based on theory of the Kurdyka-Lojasiewicz inequality, we conduct convergence analysis of these alternating iteration procedures. We particularly exemplify a family of Bernstein nonconvex penalties based on a generalized Gamma measure and conduct empirical analysis for this family.