A novel sparsity and clustering regularization
This work addresses feature selection and grouping in machine learning, but it is incremental as it builds on the existing OSCAR regularizer.
The authors tackled the problem of feature grouping in regression and classification by proposing the SPARC regularizer, which enforces sparsity and encourages non-zero features to have equal magnitude without shrinking them, and experiments on synthetic and breast cancer data showed it is a competitive group-sparsity inducing method.
We propose a novel SPARsity and Clustering (SPARC) regularizer, which is a modified version of the previous octagonal shrinkage and clustering algorithm for regression (OSCAR), where, the proposed regularizer consists of a $K$-sparse constraint and a pair-wise $\ell_{\infty}$ norm restricted on the $K$ largest components in magnitude. The proposed regularizer is able to separably enforce $K$-sparsity and encourage the non-zeros to be equal in magnitude. Moreover, it can accurately group the features without shrinking their magnitude. In fact, SPARC is closely related to OSCAR, so that the proximity operator of the former can be efficiently computed based on that of the latter, allowing using proximal splitting algorithms to solve problems with SPARC regularization. Experiments on synthetic data and with benchmark breast cancer data show that SPARC is a competitive group-sparsity inducing regularizer for regression and classification.