Faster Algorithms for High-Dimensional Robust Covariance Estimation
This work provides faster algorithms for robust covariance estimation in high dimensions, which is incremental but improves computational efficiency for applications in statistics and machine learning.
The paper tackles the problem of estimating the covariance matrix of a high-dimensional distribution when a fraction of samples are corrupted, developing an algorithm that runs in time ˜O(d^{3.26})/poly(ε) and approximates the covariance to optimal error up to a logarithmic factor, given N = ˜Ω(d^2/ε^2) samples from a Gaussian distribution.
We study the problem of estimating the covariance matrix of a high-dimensional distribution when a small constant fraction of the samples can be arbitrarily corrupted. Recent work gave the first polynomial time algorithms for this problem with near-optimal error guarantees for several natural structured distributions. Our main contribution is to develop faster algorithms for this problem whose running time nearly matches that of computing the empirical covariance. Given $N = \tildeΩ(d^2/ε^2)$ samples from a $d$-dimensional Gaussian distribution, an $ε$-fraction of which may be arbitrarily corrupted, our algorithm runs in time $\tilde{O}(d^{3.26})/\mathrm{poly}(ε)$ and approximates the unknown covariance matrix to optimal error up to a logarithmic factor. Previous robust algorithms with comparable error guarantees all have runtimes $\tildeΩ(d^{2 ω})$ when $ε= Ω(1)$, where $ω$ is the exponent of matrix multiplication. We also provide evidence that improving the running time of our algorithm may require new algorithmic techniques.