Exclusive Group Lasso for Structured Variable Selection
This work addresses variable selection problems in statistics and machine learning for researchers and practitioners, offering an incremental improvement by extending existing methods to handle more rigid sparsity patterns within groups.
The paper tackles structured variable selection with exclusive group sparsity by designing a composite norm based on atomic norms, leading to efficient algorithms like proximal and active set methods for support recovery. It demonstrates asymptotic consistency and validates results through numerical simulations, showing effectiveness in signed support recovery under conventional assumptions.
A structured variable selection problem is considered in which the covariates, divided into predefined groups, activate according to sparse patterns with few nonzero entries per group. Capitalizing on the concept of atomic norm, a composite norm can be properly designed to promote such exclusive group sparsity patterns. The resulting norm lends itself to efficient and flexible regularized optimization algorithms for support recovery, like the proximal algorithm. Moreover, an active set algorithm is proposed that builds the solution by successively including structure atoms into the estimated support. It is also shown that such an algorithm can be tailored to match more rigid structures than plain exclusive group sparsity. Asymptotic consistency analysis (with both the number of parameters as well as the number of groups growing with the observation size) establishes the effectiveness of the proposed solution in terms of signed support recovery under conventional assumptions. Finally, a set of numerical simulations further corroborates the results.