Deep Neural Tangent Kernel and Laplace Kernel Have the Same RKHS
This provides theoretical insights into kernel methods in machine learning, but it is incremental as it builds on existing kernel theory without direct empirical validation.
The paper proves that the reproducing kernel Hilbert spaces (RKHS) of the deep neural tangent kernel and the Laplace kernel contain the same functions when restricted to the sphere, and shows that the exponential power kernel with a smaller power yields a larger RKHS on the sphere and in Euclidean space.
We prove that the reproducing kernel Hilbert spaces (RKHS) of a deep neural tangent kernel and the Laplace kernel include the same set of functions, when both kernels are restricted to the sphere $\mathbb{S}^{d-1}$. Additionally, we prove that the exponential power kernel with a smaller power (making the kernel less smooth) leads to a larger RKHS, when it is restricted to the sphere $\mathbb{S}^{d-1}$ and when it is defined on the entire $\mathbb{R}^d$.