Re-Analyze Gauss: Bounds for Private Matrix Approximation via Dyson Brownian Motion
This work provides incremental improvements to differential privacy bounds for matrix problems, benefiting researchers in privacy-preserving data analysis.
The paper tackles the problem of improving utility bounds for differentially private matrix approximation by analyzing the Gaussian mechanism through the lens of Dyson Brownian motion, resulting in new bounds that depend on eigenvalue gaps and yield improvements for private rank-k covariance approximation and subspace recovery.
Given a symmetric matrix $M$ and a vector $λ$, we present new bounds on the Frobenius-distance utility of the Gaussian mechanism for approximating $M$ by a matrix whose spectrum is $λ$, under $(\varepsilon,δ)$-differential privacy. Our bounds depend on both $λ$ and the gaps in the eigenvalues of $M$, and hold whenever the top $k+1$ eigenvalues of $M$ have sufficiently large gaps. When applied to the problems of private rank-$k$ covariance matrix approximation and subspace recovery, our bounds yield improvements over previous bounds. Our bounds are obtained by viewing the addition of Gaussian noise as a continuous-time matrix Brownian motion. This viewpoint allows us to track the evolution of eigenvalues and eigenvectors of the matrix, which are governed by stochastic differential equations discovered by Dyson. These equations allow us to bound the utility as the square-root of a sum-of-squares of perturbations to the eigenvectors, as opposed to a sum of perturbation bounds obtained via Davis-Kahan-type theorems.