Kinetic Energy Plus Penalty Functions for Sparse Estimation
This work addresses sparse estimation for high-dimensional data modeling, presenting an incremental improvement by relating it to existing penalties like ℓ₁/₂ and MCP.
The authors tackled the problem of sparse estimation in high-dimensional data by proposing a new family of sparsity-inducing penalty functions called kinetic energy plus (KEP) functions, derived from the concave conjugate of a χ²-distance function, and showed that it is effective and efficient through theoretical and empirical analysis.
In this paper we propose and study a family of sparsity-inducing penalty functions. Since the penalty functions are related to the kinetic energy in special relativity, we call them \emph{kinetic energy plus} (KEP) functions. We construct the KEP function by using the concave conjugate of a $χ^2$-distance function and present several novel insights into the KEP function with $q=1$. In particular, we derive a thresholding operator based on the KEP function, and prove its mathematical properties and asymptotic properties in sparsity modeling. Moreover, we show that a coordinate descent algorithm is especially appropriate for the KEP function. Additionally, we discuss the relationship of KEP with the penalty functions $\ell_{1/2}$ and MCP. The theoretical and empirical analysis validates that the KEP function is effective and efficient in high-dimensional data modeling.