Group Regularized Estimation under Structural Hierarchy
This work addresses computational and theoretical gaps in variable selection with hierarchical constraints, which is important for statisticians and data scientists, but it is incremental as it builds on existing hierarchical methods.
The paper tackles the problem of slow convergence and lack of statistical theory in hierarchical variable selection for models with interactions, proposing a new estimator that achieves sharp rate oracle inequalities and demonstrates efficiency in simulations and real data.
Variable selection for models including interactions between explanatory variables often needs to obey certain hierarchical constraints. The weak or strong structural hierarchy requires that the existence of an interaction term implies at least one or both associated main effects to be present in the model. Lately, this problem has attracted a lot of attention, but existing computational algorithms converge slow even with a moderate number of predictors. Moreover, in contrast to the rich literature on ordinary variable selection, there is a lack of statistical theory to show reasonably low error rates of hierarchical variable selection. This work investigates a new class of estimators that make use of multiple group penalties to capture structural parsimony. We give the minimax lower bounds for strong and weak hierarchical variable selection and show that the proposed estimators enjoy sharp rate oracle inequalities. A general-purpose algorithm is developed with guaranteed convergence and global optimality. Simulations and real data experiments demonstrate the efficiency and efficacy of the proposed approach.