Invertible Kernel PCA with Random Fourier Features
This provides a more efficient alternative for denoising in kernel PCA applications, though it is incremental as it builds on existing kPCA methods.
The paper tackles the problem of reconstructing original input signals from kernel PCA for denoising by introducing invertible kernel PCA (ikPCA), which avoids solving a supervised learning problem and instead uses random Fourier features and invertibility in a subdomain, achieving similar performance to supervised methods on denoising tasks.
Kernel principal component analysis (kPCA) is a widely studied method to construct a low-dimensional data representation after a nonlinear transformation. The prevailing method to reconstruct the original input signal from kPCA -- an important task for denoising -- requires us to solve a supervised learning problem. In this paper, we present an alternative method where the reconstruction follows naturally from the compression step. We first approximate the kernel with random Fourier features. Then, we exploit the fact that the nonlinear transformation is invertible in a certain subdomain. Hence, the name \emph{invertible kernel PCA (ikPCA)}. We experiment with different data modalities and show that ikPCA performs similarly to kPCA with supervised reconstruction on denoising tasks, making it a strong alternative.