The Bernstein Function: A Unifying Framework of Nonconvex Penalization in Sparse Estimation
This work provides a theoretical framework for sparse estimation in statistics and machine learning, but it appears incremental as it builds on existing nonconvex penalty methods.
The paper tackles the problem of high-dimensional sparse estimation by introducing the Bernstein function as a unifying framework for nonconvex penalization, showing that it induces sparsity and can be optimized using coordinate descent and conjugate maximization algorithms.
In this paper we study nonconvex penalization using Bernstein functions. Since the Bernstein function is concave and nonsmooth at the origin, it can induce a class of nonconvex functions for high-dimensional sparse estimation problems. We derive a threshold function based on the Bernstein penalty and give its mathematical properties in sparsity modeling. We show that a coordinate descent algorithm is especially appropriate for penalized regression problems with the Bernstein penalty. Additionally, we prove that the Bernstein function can be defined as the concave conjugate of a $\varphi$-divergence and develop a conjugate maximization algorithm for finding the sparse solution. Finally, we particularly exemplify a family of Bernstein nonconvex penalties based on a generalized Gamma measure and conduct empirical analysis for this family.