FedSiam: Towards Adaptive Federated Semi-Supervised Learning
This work provides a general framework for federated semi-supervised learning, which is an incremental improvement for researchers and practitioners dealing with privacy-preserving machine learning with limited labeled data.
This paper addresses federated semi-supervised learning across various scenarios by proposing FedSiam, a framework built on a siamese network with momentum updates. FedSiam adaptively selects layer-level parameters for upload based on a new divergence metric, outperforming state-of-the-art baselines on three datasets under different data distribution settings.
Federated learning (FL) has emerged as an effective technique to co-training machine learning models without actually sharing data and leaking privacy. However, most existing FL methods focus on the supervised setting and ignore the utilization of unlabeled data. Although there are a few existing studies trying to incorporate unlabeled data into FL, they all fail to maintain performance guarantees or generalization ability in various real-world settings. In this paper, we focus on designing a general framework FedSiam to tackle different scenarios of federated semi-supervised learning, including four settings in the labels-at-client scenario and two setting in the labels-at-server scenario. FedSiam is built upon a siamese network into FL with a momentum update to handle the non-IID challenges introduced by unlabeled data. We further propose a new metric to measure the divergence of local model layers within the siamese network. Based on the divergence, FedSiam can automatically select layer-level parameters to be uploaded to the server in an adaptive manner. Experimental results on three datasets under two scenarios with different data distribution settings demonstrate that the proposed FedSiam framework outperforms state-of-the-art baselines.