Towards Robust Knowledge Removal in Federated Learning with High Data Heterogeneity
This addresses the need for robust knowledge removal in federated learning systems with high data heterogeneity, which is crucial for compliance with privacy regulations and maintaining service availability, representing an incremental improvement over existing methods.
The paper tackles the problem of efficiently removing a client's contribution from a federated learning model to meet privacy and safety requirements, introducing a solution based on Task Arithmetic and Neural Tangent Kernel that enables rapid removal without requiring multiple communication rounds.
Nowdays, there are an abundance of portable devices capable of collecting large amounts of data and with decent computational power. This opened the possibility to train AI models in a distributed manner, preserving the participating clients' privacy. However, because of privacy regulations and safety requirements, elimination upon necessity of a client contribution to the model has become mandatory. The cleansing process must satisfy specific efficacy and time requirements. In recent years, research efforts have produced several knowledge removal methods, but these require multiple communication rounds between the data holders and the process coordinator. This can cause the unavailability of an effective model up to the end of the removal process, which can result in a disservice to the system users. In this paper, we introduce an innovative solution based on Task Arithmetic and the Neural Tangent Kernel, to rapidly remove a client's influence from a model.