Iterative regularization for convex regularizers
This work addresses iterative regularization for researchers in optimization and machine learning, offering incremental improvements in computational efficiency for convex regularizers.
The paper tackles the problem of iterative regularization for linear models with convex but not strongly convex bias, analyzing the stability and convergence of a primal-dual gradient approach under worst-case deterministic noise, and demonstrates state-of-the-art performance with significant computational speed-ups in robust sparse recovery.
We study iterative regularization for linear models, when the bias is convex but not necessarily strongly convex. We characterize the stability properties of a primal-dual gradient based approach, analyzing its convergence in the presence of worst case deterministic noise. As a main example, we specialize and illustrate the results for the problem of robust sparse recovery. Key to our analysis is a combination of ideas from regularization theory and optimization in the presence of errors. Theoretical results are complemented by experiments showing that state-of-the-art performances can be achieved with considerable computational speed-ups.