Probing the Transition to Dataset-Level Privacy in ML Models Using an Output-Specific and Data-Resolved Privacy Profile
This work addresses the practical challenge for ML practitioners in implementing differential privacy by providing tools to visualize and select privacy parameters, though it is incremental as it builds on existing DP frameworks.
The paper tackles the problem of selecting the privacy budget ε and quantifying individual privacy leakage in differentially private machine learning models by introducing a privacy profile that ranks the privacy of training samples and probes a transition to indistinguishability as ε decreases, enabling practitioners to choose ε based on observed coverage metrics.
Differential privacy (DP) is the prevailing technique for protecting user data in machine learning models. However, deficits to this framework include a lack of clarity for selecting the privacy budget $ε$ and a lack of quantification for the privacy leakage for a particular data row by a particular trained model. We make progress toward these limitations and a new perspective by which to visualize DP results by studying a privacy metric that quantifies the extent to which a model trained on a dataset using a DP mechanism is ``covered" by each of the distributions resulting from training on neighboring datasets. We connect this coverage metric to what has been established in the literature and use it to rank the privacy of individual samples from the training set in what we call a privacy profile. We additionally show that the privacy profile can be used to probe an observed transition to indistinguishability that takes place in the neighboring distributions as $ε$ decreases, which we suggest is a tool that can enable the selection of $ε$ by the ML practitioner wishing to make use of DP.