Distributed Differentially Private Mutual Information Ranking and Its Applications
This addresses privacy concerns for organizations handling sensitive data like user information, though it is incremental as it builds on existing mutual information and differential privacy methods.
The paper tackled the privacy risks of mutual information ranking on large sensitive datasets by introducing Distributed Differentially Private Mutual Information (DDP-MI), which provides strong privacy protections and substantially improves efficiency compared to standard implementations.
Computation of Mutual Information (MI) helps understand the amount of information shared between a pair of random variables. Automated feature selection techniques based on MI ranking are regularly used to extract information from sensitive datasets exceeding petabytes in size, over millions of features and classes. Series of one-vs-all MI computations can be cascaded to produce n-fold MI results, rapidly pinpointing informative relationships. This ability to quickly pinpoint the most informative relationships from datasets of billions of users creates privacy concerns. In this paper, we present Distributed Differentially Private Mutual Information (DDP-MI), a privacy-safe fast batch MI, across various scenarios such as feature selection, segmentation, ranking, and query expansion. This distributed implementation is protected with global model differential privacy to provide strong assurances against a wide range of privacy attacks. We also show that our DDP-MI can substantially improve the efficiency of MI calculations compared to standard implementations on a large-scale public dataset.