Estimating covariance and precision matrices along subspaces
This addresses problems in dimension reduction and structured regression where data behavior along lower-dimensional spaces is critical, though it appears incremental in extending existing estimation theory.
The paper analyzes how accurately covariance and precision matrices can be estimated along specific subspaces or directions using finite samples, showing that accuracy depends mainly on components in those subspaces and that precision matrix estimation is largely unaffected by covariance condition number. It proposes a new estimator for single-index models with strong theoretical guarantees and superior numerical performance.
We study the accuracy of estimating the covariance and the precision matrix of a $D$-variate sub-Gaussian distribution along a prescribed subspace or direction using the finite sample covariance. Our results show that the estimation accuracy depends almost exclusively on the components of the distribution that correspond to desired subspaces or directions. This is relevant and important for problems where the behavior of data along a lower-dimensional space is of specific interest, such as dimension reduction or structured regression problems. We also show that estimation of precision matrices is almost independent of the condition number of the covariance matrix. The presented applications include direction-sensitive eigenspace perturbation bounds, relative bounds for the smallest eigenvalue, and the estimation of the single-index model. For the latter, a new estimator, derived from the analysis, with strong theoretical guarantees and superior numerical performance is proposed.