Asymptotic behavior of eigenvalues of large rank perturbations of large random matrices
This work addresses a theoretical gap in random matrix theory for machine learning applications, but it is incremental as it extends prior finite-rank results to growing-rank scenarios.
The paper tackles the problem of analyzing eigenvalues for large random matrices with growing-rank perturbations, which is relevant for understanding weight matrices in trained deep neural networks, and develops asymptotic analysis for this case.
The paper is concerned with deformed Wigner random matrices. These matrices are closely connected with Deep Neural Networks (DNNs): weight matrices of trained DNNs could be represented in the form $R + S$, where $R$ is random and $S$ is highly correlated. The spectrum of such matrices plays a key role in rigorous underpinning of the novel pruning technique based on Random Matrix Theory. Mathematics has been done only for finite-rank matrix $S$. However, in practice rank may grow. In this paper we develop asymptotic analysis for the case of growing rank.