Interpret Federated Learning with Shapley Values
This addresses the need for model interpretability in privacy-sensitive federated learning settings, particularly for vertical data splits, but is incremental as it adapts existing Shapley value techniques to this context.
The paper tackles the problem of interpreting vertical Federated Learning models while protecting data privacy, proposing a method using Shapley values to reveal detailed feature importance for host features and a unified importance for guest features, with experiments showing robust and informative results.
Federated Learning is introduced to protect privacy by distributing training data into multiple parties. Each party trains its own model and a meta-model is constructed from the sub models. In this way the details of the data are not disclosed in between each party. In this paper we investigate the model interpretation methods for Federated Learning, specifically on the measurement of feature importance of vertical Federated Learning where feature space of the data is divided into two parties, namely host and guest. For host party to interpret a single prediction of vertical Federated Learning model, the interpretation results, namely the feature importance, are very likely to reveal the protected data from guest party. We propose a method to balance the model interpretability and data privacy in vertical Federated Learning by using Shapley values to reveal detailed feature importance for host features and a unified importance value for federated guest features. Our experiments indicate robust and informative results for interpreting Federated Learning models.