Lottery Tickets in Linear Models: An Analysis of Iterative Magnitude Pruning
This work provides theoretical insights into pruning methods for linear models, which is incremental to existing research on the lottery ticket hypothesis.
The paper analyzes iterative magnitude pruning (IMP) in linear models trained by gradient flow, identifying conditions under which IMP prunes features with the smallest projection onto the data and explores its use for sparse estimation.
We analyse the pruning procedure behind the lottery ticket hypothesis arXiv:1803.03635v5, iterative magnitude pruning (IMP), when applied to linear models trained by gradient flow. We begin by presenting sufficient conditions on the statistical structure of the features under which IMP prunes those features that have smallest projection onto the data. Following this, we explore IMP as a method for sparse estimation.