Kernel PCA with the Nyström method
This work addresses the problem of making kernel PCA scalable for practitioners in machine learning, though it is incremental as it adapts an existing technique to a specific method.
The paper tackles the scalability issue of kernel PCA by deriving it with the Nyström method, providing a scalable alternative and analyzing its statistical accuracy with a finite-sample confidence bound on reconstruction error, validated through experiments on real-world datasets.
The Nyström method is one of the most popular techniques for improving the scalability of kernel methods. However, it has not yet been derived for kernel PCA in line with classical PCA. In this paper we derive kernel PCA with the Nyström method, thereby providing one of the few available options to make kernel PCA scalable. We further study its statistical accuracy through a finite-sample confidence bound on the empirical reconstruction error compared to the full method. The behaviours of the method and bound are illustrated through computer experiments on multiple real-world datasets. As an application of the method we present kernel principal component regression with the Nyström method, as an alternative to Nyström kernel ridge regression for efficient regularized regression with kernels.