Adaptive multi-penalty regularization based on a generalized Lasso path
This addresses a tedious and critical problem for researchers and practitioners in machine learning and signal processing dealing with undetermined sparse regression, though it appears incremental as it builds upon existing single-penalty theories.
The paper tackles the challenge of parameter tuning in multi-penalty regularization for sparse regression, proposing an adaptive algorithmic framework that efficiently constructs solution regions and selects parameters to improve support recovery, with numerical analysis showing robustness compared to single-penalty methods.
For many algorithms, parameter tuning remains a challenging and critical task, which becomes tedious and infeasible in a multi-parameter setting. Multi-penalty regularization, successfully used for solving undetermined sparse regression of problems of unmixing type where signal and noise are additively mixed, is one of such examples. In this paper, we propose a novel algorithmic framework for an adaptive parameter choice in multi-penalty regularization with a focus on the correct support recovery. Building upon the theory of regularization paths and algorithms for single-penalty functionals, we extend these ideas to a multi-penalty framework by providing an efficient procedure for the construction of regions containing structurally similar solutions, i.e., solutions with the same sparsity and sign pattern, over the whole range of parameters. Combining this with a model selection criterion, we can choose regularization parameters in a data-adaptive manner. Another advantage of our algorithm is that it provides an overview on the solution stability over the whole range of parameters. This can be further exploited to obtain additional insights into the problem of interest. We provide a numerical analysis of our method and compare it to the state-of-the-art single-penalty algorithms for compressed sensing problems in order to demonstrate the robustness and power of the proposed algorithm.