On the Approximation of Kernel functions
This work addresses kernel approximation challenges in statistical learning, offering incremental improvements for methods like Nyström by enabling more efficient low-rank approximations.
The paper tackles the problem of approximating kernel functions in high-dimensional compact sets by proposing Taylor series approximations for radial kernels, specifically establishing an upper bound for Gauss kernel eigenfunctions that grows polynomially with index, leading to smaller regularization parameters and better approximations.
Various methods in statistical learning build on kernels considered in reproducing kernel Hilbert spaces. In applications, the kernel is often selected based on characteristics of the problem and the data. This kernel is then employed to infer response variables at points, where no explanatory data were observed. The data considered here are located in compact sets in higher dimensions and the paper addresses approximations of the kernel itself. The new approach considers Taylor series approximations of radial kernel functions. For the Gauss kernel on the unit cube, the paper establishes an upper bound of the associated eigenfunctions, which grows only polynomially with respect to the index. The novel approach substantiates smaller regularization parameters than considered in the literature, overall leading to better approximations. This improvement confirms low rank approximation methods such as the Nyström method.