Nonconvex Sparse Learning via Stochastic Optimization with Progressive Variance Reduction
This work addresses high-dimensional sparse learning problems for machine learning and statistics applications, presenting an incremental improvement with a novel method for a known bottleneck.
The authors tackled sparse learning problems with cardinality constraints by developing a stochastic variance reduced optimization algorithm that achieves strong linear convergence guarantees and optimal estimation accuracy in high dimensions, with numerical experiments showing efficiency in parameter estimation and computational performance.
We propose a stochastic variance reduced optimization algorithm for solving sparse learning problems with cardinality constraints. Sufficient conditions are provided, under which the proposed algorithm enjoys strong linear convergence guarantees and optimal estimation accuracy in high dimensions. We further extend the proposed algorithm to an asynchronous parallel variant with a near linear speedup. Numerical experiments demonstrate the efficiency of our algorithm in terms of both parameter estimation and computational performance.