Large-scale Nonlinear Variable Selection via Kernel Random Features
This addresses the challenge of scaling kernel methods for variable selection in large-scale nonlinear regression, which is incremental as it builds on existing kernel and random feature techniques.
The authors tackled the problem of input variable selection in nonlinear regression for large datasets by proposing a kernel-based method using random features, achieving outstanding performance on synthetic and real datasets.
We propose a new method for input variable selection in nonlinear regression. The method is embedded into a kernel regression machine that can model general nonlinear functions, not being a priori limited to additive models. This is the first kernel-based variable selection method applicable to large datasets. It sidesteps the typical poor scaling properties of kernel methods by mapping the inputs into a relatively low-dimensional space of random features. The algorithm discovers the variables relevant for the regression task together with learning the prediction model through learning the appropriate nonlinear random feature maps. We demonstrate the outstanding performance of our method on a set of large-scale synthetic and real datasets.