Don't Fall for Tuning Parameters: Tuning-Free Variable Selection in High Dimensions With the TREX
This addresses a practical bottleneck for statisticians and data scientists using high-dimensional models by offering a tuning-free method, though it is incremental as it builds on Lasso and Square-Root Lasso.
The paper tackles the problem of tuning parameter calibration in high-dimensional variable selection by introducing TREX, a Lasso alternative that inherently calibrates to all model aspects without tuning parameters, showing it outperforms cross-validated Lasso in variable selection and computational efficiency on synthetic and biological data.
Lasso is a seminal contribution to high-dimensional statistics, but it hinges on a tuning parameter that is difficult to calibrate in practice. A partial remedy for this problem is Square-Root Lasso, because it inherently calibrates to the noise variance. However, Square-Root Lasso still requires the calibration of a tuning parameter to all other aspects of the model. In this study, we introduce TREX, an alternative to Lasso with an inherent calibration to all aspects of the model. This adaptation to the entire model renders TREX an estimator that does not require any calibration of tuning parameters. We show that TREX can outperform cross-validated Lasso in terms of variable selection and computational efficiency. We also introduce a bootstrapped version of TREX that can further improve variable selection. We illustrate the promising performance of TREX both on synthetic data and on a recent high-dimensional biological data set that considers riboflavin production in B. subtilis.