Efficient Path Algorithms for Clustered Lasso and OSCAR
This work addresses a computational bottleneck for researchers and practitioners using feature clustering in regression, though it is incremental as it improves upon existing algorithms rather than introducing a new paradigm.
The paper tackled the computational inefficiency of clustered Lasso and OSCAR in high-dimensional regression by proposing efficient path algorithms that reduce costs using symmetry and graph theory, achieving faster performance than existing methods in numerical experiments.
In high dimensional regression, feature clustering by their effects on outcomes is often as important as feature selection. For that purpose, clustered Lasso and octagonal shrinkage and clustering algorithm for regression (OSCAR) are used to make feature groups automatically by pairwise $L_1$ norm and pairwise $L_\infty$ norm, respectively. This paper proposes efficient path algorithms for clustered Lasso and OSCAR to construct solution paths with respect to their regularization parameters. Despite too many terms in exhaustive pairwise regularization, their computational costs are reduced by using symmetry of those terms. Simple equivalent conditions to check subgradient equations in each feature group are derived by some graph theories. The proposed algorithms are shown to be more efficient than existing algorithms in numerical experiments.