Ensemble Kernel Methods, Implicit Regularization and Determinantal Point Processes
This work addresses regularization challenges in kernel methods for machine learning practitioners, but it appears incremental as it builds on existing DPP frameworks.
The paper tackles the problem of implicit regularization in ridgeless kernel regression by using Determinantal Point Processes (DPPs) to sample diverse subsets, showing that ensembles of these regressors can be effective for datasets with redundant information.
By using the framework of Determinantal Point Processes (DPPs), some theoretical results concerning the interplay between diversity and regularization can be obtained. In this paper we show that sampling subsets with kDPPs results in implicit regularization in the context of ridgeless Kernel Regression. Furthermore, we leverage the common setup of state-of-the-art DPP algorithms to sample multiple small subsets and use them in an ensemble of ridgeless regressions. Our first empirical results indicate that ensemble of ridgeless regressors can be interesting to use for datasets including redundant information.