Gen-Oja: A Two-time-scale approach for Streaming CCA
This work addresses efficient streaming data analysis for machine learning applications, representing an incremental improvement with specific algorithmic enhancements.
The paper tackles the problems of principal Generalized Eigenvector computation and Canonical Correlation Analysis in stochastic settings by proposing the Gen-Oja algorithm, which achieves optimal convergence rates with proven global convergence.
In this paper, we study the problems of principal Generalized Eigenvector computation and Canonical Correlation Analysis in the stochastic setting. We propose a simple and efficient algorithm, Gen-Oja, for these problems. We prove the global convergence of our algorithm, borrowing ideas from the theory of fast-mixing Markov chains and two-time-scale stochastic approximation, showing that it achieves the optimal rate of convergence. In the process, we develop tools for understanding stochastic processes with Markovian noise which might be of independent interest.