Robust Grouped Variable Selection Using Distributionally Robust Optimization
This work addresses robust variable selection for statistical modeling in noisy environments like medical records, representing an incremental improvement over existing GLASSO methods.
The authors tackled the problem of selecting grouped variables in regression and classification under data perturbations by proposing a Distributionally Robust Optimization formulation with Wasserstein-based uncertainty sets. Their approach achieved better prediction and estimation performance in the presence of outliers compared to alternatives, as demonstrated on synthetic data and a large medical dataset.
We propose a Distributionally Robust Optimization (DRO) formulation with a Wasserstein-based uncertainty set for selecting grouped variables under perturbations on the data for both linear regression and classification problems. The resulting model offers robustness explanations for Grouped Least Absolute Shrinkage and Selection Operator (GLASSO) algorithms and highlights the connection between robustness and regularization. We prove probabilistic bounds on the out-of-sample loss and the estimation bias, and establish the grouping effect of our estimator, showing that coefficients in the same group converge to the same value as the sample correlation between covariates approaches 1. Based on this result, we propose to use the spectral clustering algorithm with the Gaussian similarity function to perform grouping on the predictors, which makes our approach applicable without knowing the grouping structure a priori. We compare our approach to an array of alternatives and provide extensive numerical results on both synthetic data and a real large dataset of surgery-related medical records, showing that our formulation produces an interpretable and parsimonious model that encourages sparsity at a group level and is able to achieve better prediction and estimation performance in the presence of outliers.