Don't relax: early stopping for convex regularization
This work addresses regularization efficiency for machine learning practitioners, offering a novel approach that is incremental in improving computational aspects.
The paper tackles the problem of efficient regularization in convex settings by proposing an early stopping method instead of traditional penalization, achieving comparable recovery accuracy while incorporating computational efficiency.
We consider the problem of designing efficient regularization algorithms when regularization is encoded by a (strongly) convex functional. Unlike classical penalization methods based on a relaxation approach, we propose an iterative method where regularization is achieved via early stopping. Our results show that the proposed procedure achieves the same recovery accuracy as penalization methods, while naturally integrating computational considerations. An empirical analysis on a number of problems provides promising results with respect to the state of the art.