Subsampling for supervised learning in reproducing kernel Hilbert spaces
For practitioners in statistical learning, this work provides a principled subsampling method to reduce computational burden and energy consumption in nonparametric settings.
This paper proposes an optimal subsampling scheme for supervised learning in reproducing kernel Hilbert spaces, reducing computational cost while maintaining asymptotic properties. Numerical experiments demonstrate its practicality and benefit.
In the era of big data, subsampling became a common practice in statistical learning. By selecting a subgroup of individuals based on which the learner is trained, subsampling aims at reducing the computational cost and time of the estimation step, and ideally leads to a decrease of its energy consumption and carbon footprint. This work focuses on a nonparametric setting, in which the hypotheses set lies in a reproducing kernel Hilbert space, and the estimator is a minimizer of an empirical risk reweighted à la Horvitz-Thompson. By studying the asymptotic properties of this estimator, we reveal an optimal subsampling scheme (regarding the trace of the covariance operator) and show that it can be used via plug-in. A numerical study on synthetic and real-world datasets shows the practicability and the benefit of the proposed approach.