FedPH: Privacy-enhanced Heterogeneous Federated Learning
This work addresses privacy and heterogeneity challenges in federated learning for distributed machine learning applications, representing an incremental improvement over existing methods.
The paper tackles the problem of heterogeneous data distributions and computing resources in federated learning by proposing a method that uses a pre-trained backbone with shared class embedding vectors and adds noise for privacy, achieving improved performance and effective communication with minimal impact under differential privacy on a self-built vehicle dataset.
Federated Learning is a distributed machine-learning environment that allows clients to learn collaboratively without sharing private data. This is accomplished by exchanging parameters. However, the differences in data distributions and computing resources among clients make related studies difficult. To address these heterogeneous problems, we propose a novel Federated Learning method. Our method utilizes a pre-trained model as the backbone of the local model, with fully connected layers comprising the head. The backbone extracts features for the head, and the embedding vector of classes is shared between clients to improve the head and enhance the performance of the local model. By sharing the embedding vector of classes instead of gradient-based parameters, clients can better adapt to private data, and communication between the server and clients is more effective. To protect privacy, we propose a privacy-preserving hybrid method that adds noise to the embedding vector of classes. This method has a minimal effect on the performance of the local model when differential privacy is met. We conduct a comprehensive evaluation of our approach on a self-built vehicle dataset, comparing it with other Federated Learning methods under non-independent identically distributed(Non-IID).