Fast sparse optimization via adaptive shrinkage
This incremental improvement addresses the need for faster sparse optimization in large-dimensional data-driven problems and time-varying systems.
The paper tackles the slow convergence of iterative shrinkage-thresholding algorithms for Lasso in sparse optimization by developing a proximal method with adaptive shrinkage hyperparameter, resulting in faster convergence while maintaining simplicity, as validated by numerical experiments.
The need for fast sparse optimization is emerging, e.g., to deal with large-dimensional data-driven problems and to track time-varying systems. In the framework of linear sparse optimization, the iterative shrinkage-thresholding algorithm is a valuable method to solve Lasso, which is particularly appreciated for its ease of implementation. Nevertheless, it converges slowly. In this paper, we develop a proximal method, based on logarithmic regularization, which turns out to be an iterative shrinkage-thresholding algorithm with adaptive shrinkage hyperparameter. This adaptivity substantially enhances the trajectory of the algorithm, in a way that yields faster convergence, while keeping the simplicity of the original method. Our contribution is twofold: on the one hand, we derive and analyze the proposed algorithm; on the other hand, we validate its fast convergence via numerical experiments and we discuss the performance with respect to state-of-the-art algorithms.