Learning Model-Based Sparsity via Projected Gradient Descent
This work addresses the challenge of parameter tuning and model adherence in structured sparsity estimation for statistical modeling, offering a more reliable alternative to convex methods.
The paper tackles the problem of structured sparsity estimation by proposing a projected gradient descent method with non-convex constraints, which avoids the need for careful regularization tuning and ensures estimates belong to the sparsity model, and demonstrates its application to Generalized Linear Models.
Several convex formulation methods have been proposed previously for statistical estimation with structured sparsity as the prior. These methods often require a carefully tuned regularization parameter, often a cumbersome or heuristic exercise. Furthermore, the estimate that these methods produce might not belong to the desired sparsity model, albeit accurately approximating the true parameter. Therefore, greedy-type algorithms could often be more desirable in estimating structured-sparse parameters. So far, these greedy methods have mostly focused on linear statistical models. In this paper we study the projected gradient descent with non-convex structured-sparse parameter model as the constraint set. Should the cost function have a Stable Model-Restricted Hessian the algorithm produces an approximation for the desired minimizer. As an example we elaborate on application of the main results to estimation in Generalized Linear Model.