A Spectral Analysis of Dot-product Kernels
This work addresses theoretical analysis for kernel methods in machine learning, focusing on compositional structures, but appears incremental as it builds on prior estimates for specific kernels.
The paper tackled eigenvalue decay estimates for compositional dot-product kernels, improving previous bounds for power series kernels on spheres, which enabled calculation of reproducing kernel Hilbert space ball volumes and revealed trade-offs between approximation and statistical errors based on composition count and kernel smoothness.
We present eigenvalue decay estimates of integral operators associated with compositional dot-product kernels. The estimates improve on previous ones established for power series kernels on spheres. This allows us to obtain the volumes of balls in the corresponding reproducing kernel Hilbert spaces. We discuss the consequences on statistical estimation with compositional dot product kernels and highlight interesting trade-offs between the approximation error and the statistical error depending on the number of compositions and the smoothness of the kernels.