Scalable Subset Selection in Linear Mixed Models
This addresses a bottleneck in personalized medicine and other fields with heterogeneous data, offering a scalable solution for subset selection in LMMs, though it is incremental as it builds on existing sparse methods.
The paper tackles the scalability issue of sparse learning methods for linear mixed models (LMMs) by introducing a new ℓ0 regularized subset selection method that can handle thousands of predictors in seconds to minutes, demonstrating excellent performance in experiments.
Linear mixed models (LMMs), which incorporate fixed and random effects, are key tools for analyzing heterogeneous data, such as in personalized medicine. Nowadays, this type of data is increasingly wide, sometimes containing thousands of candidate predictors, necessitating sparsity for prediction and interpretation. However, existing sparse learning methods for LMMs do not scale well beyond tens or hundreds of predictors, leaving a large gap compared with sparse methods for linear models, which ignore random effects. This paper closes the gap with a new $\ell_0$ regularized method for LMM subset selection that can run on datasets containing thousands of predictors in seconds to minutes. On the computational front, we develop a coordinate descent algorithm as our main workhorse and provide a guarantee of its convergence. We also develop a local search algorithm to help traverse the nonconvex optimization surface. Both algorithms readily extend to subset selection in generalized LMMs via a penalized quasi-likelihood approximation. On the statistical front, we provide a finite-sample bound on the Kullback-Leibler divergence of the new method. We then demonstrate its excellent performance in experiments involving synthetic and real datasets.