Adaptive Iterative Soft-Thresholding Algorithm with the Median Absolute Deviation
This provides theoretical foundations for a practical algorithm in sparse regression, but it is incremental as it builds on existing adaptive ISTA methods.
The paper tackles the lack of theoretical results for the adaptive Iterative Soft-Thresholding Algorithm (ISTA) used in LASSO problems, by analyzing its properties with a median absolute deviation thresholding strategy, proving local linear convergence and global convergence behavior.
The adaptive Iterative Soft-Thresholding Algorithm (ISTA) has been a popular algorithm for finding a desirable solution to the LASSO problem without explicitly tuning the regularization parameter $λ$. Despite that the adaptive ISTA is a successful practical algorithm, few theoretical results exist. In this paper, we present the theoretical analysis on the adaptive ISTA with the thresholding strategy of estimating noise level by median absolute deviation. We show properties of the fixed points of the algorithm, including scale equivariance, non-uniqueness, and local stability, prove the local linear convergence guarantee, and show its global convergence behavior.