Sobolev norm inconsistency of kernel interpolation
This addresses a foundational issue in machine learning theory for researchers, revealing inherent limitations in kernel methods.
The paper tackles the problem of consistency in kernel interpolation by deriving lower bounds for generalization error across a range of norms, showing that kernel interpolation is always inconsistent when the smoothness index exceeds a threshold dependent on embedding and eigenvalue decay.
We study the consistency of minimum-norm interpolation in reproducing kernel Hilbert spaces corresponding to bounded kernels. Our main result give lower bounds for the generalization error of the kernel interpolation measured in a continuous scale of norms that interpolate between $L^2$ and the hypothesis space. These lower bounds imply that kernel interpolation is always inconsistent, when the smoothness index of the norm is larger than a constant that depends only on the embedding index of the hypothesis space and the decay rate of the eigenvalues.