Private Matrix Approximation and Geometry of Unitary Orbits
This addresses privacy concerns in matrix approximation for users' data, offering incremental improvements over existing methods.
The paper tackles the problem of designing differentially private algorithms for matrix approximation by optimizing over unitary orbits, with applications like PCA, and provides efficient algorithms with error bounds that unify and improve prior work.
Consider the following optimization problem: Given $n \times n$ matrices $A$ and $Λ$, maximize $\langle A, UΛU^*\rangle$ where $U$ varies over the unitary group $\mathrm{U}(n)$. This problem seeks to approximate $A$ by a matrix whose spectrum is the same as $Λ$ and, by setting $Λ$ to be appropriate diagonal matrices, one can recover matrix approximation problems such as PCA and rank-$k$ approximation. We study the problem of designing differentially private algorithms for this optimization problem in settings where the matrix $A$ is constructed using users' private data. We give efficient and private algorithms that come with upper and lower bounds on the approximation error. Our results unify and improve upon several prior works on private matrix approximation problems. They rely on extensions of packing/covering number bounds for Grassmannians to unitary orbits which should be of independent interest.