Hybrid safe-strong rules for efficient optimization in lasso-type problems
This work addresses the Big Data challenge for researchers and practitioners in data mining, machine learning, and high-dimensional statistics by providing more efficient optimization algorithms for lasso-type problems.
The paper tackles the computational inefficiency of lasso-type models in high-dimensional data by proposing hybrid safe-strong rules (HSSR), which combine safe screening with sequential strong rules to remove unnecessary features, resulting in substantial performance improvements over existing state-of-the-art rules in synthetic and real data experiments.
The lasso model has been widely used for model selection in data mining, machine learning, and high-dimensional statistical analysis. However, with the ultrahigh-dimensional, large-scale data sets now collected in many real-world applications, it is important to develop algorithms to solve the lasso that efficiently scale up to problems of this size. Discarding features from certain steps of the algorithm is a powerful technique for increasing efficiency and addressing the Big Data challenge. In this paper, we propose a family of hybrid safe-strong rules (HSSR) which incorporate safe screening rules into the sequential strong rule (SSR) to remove unnecessary computational burden. In particular, we present two instances of HSSR, namely SSR-Dome and SSR-BEDPP, for the standard lasso problem. We further extend SSR-BEDPP to the elastic net and group lasso problems to demonstrate the generalizability of the hybrid screening idea. Extensive numerical experiments with synthetic and real data sets are conducted for both the standard lasso and the group lasso problems. Results show that our proposed hybrid rules can substantially outperform existing state-of-the-art rules.