An Exact Solution Path Algorithm for SLOPE and Quasi-Spherical OSCAR
This work addresses tuning parameter sensitivity in high-dimensional regression for statisticians and data scientists, representing an incremental improvement in regularization methods.
The authors tackled the sensitivity of feature selection and clustering in SLOPE regularization to tuning parameters by developing an exact solution path algorithm and proposing QS-OSCAR, a new regularization sequence design, which simulation studies showed performs feature clustering more efficiently than other designs.
Sorted $L_1$ penalization estimator (SLOPE) is a regularization technique for sorted absolute coefficients in high-dimensional regression. By arbitrarily setting its regularization weights $λ$ under the monotonicity constraint, SLOPE can have various feature selection and clustering properties. On weight tuning, the selected features and their clusters are very sensitive to the tuning parameters. Moreover, the exhaustive tracking of their changes is difficult using grid search methods. This study presents a solution path algorithm that provides the complete and exact path of solutions for SLOPE in fine-tuning regularization weights. A simple optimality condition for SLOPE is derived and used to specify the next splitting point of the solution path. This study also proposes a new design of a regularization sequence $λ$ for feature clustering, which is called the quasi-spherical and octagonal shrinkage and clustering algorithm for regression (QS-OSCAR). QS-OSCAR is designed with a contour surface of the regularization terms most similar to a sphere. Among several regularization sequence designs, sparsity and clustering performance are compared through simulation studies. The numerical observations show that QS-OSCAR performs feature clustering more efficiently than other designs.