Empirical Evaluation of Kernel PCA Approximation Methods in Classification Tasks
This work addresses the scalability issue of KPCA for practitioners in machine learning, but it is incremental as it empirically compares existing methods without introducing new ones.
The paper tackled the problem of evaluating kernel PCA approximation methods for classification tasks, finding that Streaming Kernel PCA (SKPCA) achieved much better accuracy than other methods on a very large dataset.
Kernel Principal Component Analysis (KPCA) is a popular dimensionality reduction technique with a wide range of applications. However, it suffers from the problem of poor scalability. Various approximation methods have been proposed in the past to overcome this problem. The Nyström method, Randomized Nonlinear Component Analysis (RNCA) and Streaming Kernel Principal Component Analysis (SKPCA) were proposed to deal with the scalability issue of KPCA. Despite having theoretical guarantees, their performance in real world learning tasks have not been explored previously. In this work the evaluation of SKPCA, RNCA and Nyström method for the task of classification is done for several real world datasets. The results obtained indicate that SKPCA based features gave much better classification accuracy when compared to the other methods for a very large dataset.