Singular Value Decomposition of Operators on Reproducing Kernel Hilbert Spaces
This provides theoretical underpinnings for kernel methods used in statistics and machine learning, though it appears incremental as it extends existing eigenvalue decomposition approaches.
The paper establishes a functional analytic foundation for eigenvalue decomposition of reproducing kernel Hilbert space operators and extends this approach to singular value decomposition, providing theoretical support for applications in kernel-based machine learning methods.
Reproducing kernel Hilbert spaces (RKHSs) play an important role in many statistics and machine learning applications ranging from support vector machines to Gaussian processes and kernel embeddings of distributions. Operators acting on such spaces are, for instance, required to embed conditional probability distributions in order to implement the kernel Bayes rule and build sequential data models. It was recently shown that transfer operators such as the Perron-Frobenius or Koopman operator can also be approximated in a similar fashion using covariance and cross-covariance operators and that eigenfunctions of these operators can be obtained by solving associated matrix eigenvalue problems. The goal of this paper is to provide a solid functional analytic foundation for the eigenvalue decomposition of RKHS operators and to extend the approach to the singular value decomposition. The results are illustrated with simple guiding examples.