Convergence Rate of Krasulina Estimator
This is an incremental theoretical analysis of an existing estimator for principal component analysis, relevant for statisticians and machine learning researchers.
The paper tackles the problem of estimating the least eigenvalue and eigenvector of a covariance matrix using the Krasulina estimator, providing a convergence proof and deriving its convergence rate.
Principal component analysis (PCA) is one of the most commonly used statistical procedures with a wide range of applications. Consider the points $X_1, X_2,..., X_n$ are vectors drawn i.i.d. from a distribution with mean zero and covariance $Σ$, where $Σ$ is unknown. Let $A_n = X_nX_n^T$, then $E[A_n] = Σ$. This paper consider the problem of finding the least eigenvalue and eigenvector of matrix $Σ$. A classical such estimator are due to Krasulina\cite{krasulina_method_1969}. We are going to state the convergence proof of Krasulina for the least eigenvalue and corresponding eigenvector, and then find their convergence rate.