Nyström landmark sampling and regularized Christoffel functions
This addresses the bottleneck of handling large training sets in kernel methods, offering an incremental improvement over existing landmark selection techniques.
The paper tackles the problem of selecting diverse and important landmarks from large datasets for kernel methods, proposing deterministic and randomized adaptive algorithms based on kernelized Christoffel functions, with results showing improved accuracy in Kernel Ridge Regression.
Selecting diverse and important items, called landmarks, from a large set is a problem of interest in machine learning. As a specific example, in order to deal with large training sets, kernel methods often rely on low rank matrix Nyström approximations based on the selection or sampling of landmarks. In this context, we propose a deterministic and a randomized adaptive algorithm for selecting landmark points within a training data set. These landmarks are related to the minima of a sequence of kernelized Christoffel functions. Beyond the known connection between Christoffel functions and leverage scores, a connection of our method with finite determinantal point processes (DPPs) is also explained. Namely, our construction promotes diversity among important landmark points in a way similar to DPPs. Also, we explain how our randomized adaptive algorithm can influence the accuracy of Kernel Ridge Regression.