One-shot Distibuted Algorithm for PCA with RBF Kernels
This work addresses distributed machine learning for large-scale data, but it is incremental as it adapts existing ideas from sample-distributed scenarios to feature-distributed cases.
The paper tackles the problem of performing kernel PCA in a feature-distributed setting by proposing a one-shot algorithm, achieving high-quality results with low communication cost when eigenvalues decay rapidly.
This letter proposes a one-shot algorithm for feature-distributed kernel PCA. Our algorithm is inspired by the dual relationship between sample-distributed and feature-distributed scenario. This interesting relationship makes it possible to establish distributed kernel PCA for feature-distributed cases from ideas in distributed PCA in sample-distributed scenario. In theoretical part, we analyze the approximation error for both linear and RBF kernels. The result suggests that when eigenvalues decay fast, the proposed algorithm gives high quality results with low communication cost. This result is also verified by numerical experiments, showing the effectiveness of our algorithm in practice.