Extending Kernel PCA through Dualization: Sparsity, Robustness and Fast Algorithms
This work addresses efficiency and flexibility limitations in KPCA for machine learning practitioners, though it is an incremental extension of existing methods.
The paper tackles the computational expense and lack of robustness/sparsity in Kernel Principal Component Analysis (KPCA) by dualizing a difference of convex functions, enabling efficient gradient-based algorithms that avoid costly SVD. Results show significant speedup in training time and improved robustness and sparsity on synthetic and real-world benchmarks.
The goal of this paper is to revisit Kernel Principal Component Analysis (KPCA) through dualization of a difference of convex functions. This allows to naturally extend KPCA to multiple objective functions and leads to efficient gradient-based algorithms avoiding the expensive SVD of the Gram matrix. Particularly, we consider objective functions that can be written as Moreau envelopes, demonstrating how to promote robustness and sparsity within the same framework. The proposed method is evaluated on synthetic and real-world benchmarks, showing significant speedup in KPCA training time as well as highlighting the benefits in terms of robustness and sparsity.