Model Explanations with Differential Privacy
This work addresses privacy concerns in algorithmic transparency for critical decision-making domains, offering a novel method to protect training and explanation data.
The authors tackled the problem of model explanations leaking sensitive information by proposing differentially private algorithms for generating feature-based explanations, achieving minimal privacy budget while maintaining explanation accuracy.
Black-box machine learning models are used in critical decision-making domains, giving rise to several calls for more algorithmic transparency. The drawback is that model explanations can leak information about the training data and the explanation data used to generate them, thus undermining data privacy. To address this issue, we propose differentially private algorithms to construct feature-based model explanations. We design an adaptive differentially private gradient descent algorithm, that finds the minimal privacy budget required to produce accurate explanations. It reduces the overall privacy loss on explanation data, by adaptively reusing past differentially private explanations. It also amplifies the privacy guarantees with respect to the training data. We evaluate the implications of differentially private models and our privacy mechanisms on the quality of model explanations.