Federated Semi-Supervised Domain Adaptation via Knowledge Transfer
This addresses the challenge of adapting models across domains with limited labeled data while preserving privacy in distributed environments, representing an incremental advancement by combining existing techniques.
The paper tackles the problem of semi-supervised domain adaptation (SSDA) in distributed settings where data cannot be shared due to privacy concerns, proposing Federated Semi-Supervised Domain Adaptation (FSSDA) that integrates SSDA with federated learning and knowledge distillation, achieving improved efficiency and generalization to multi-source scenarios as demonstrated in experiments.
Given the rapidly changing machine learning environments and expensive data labeling, semi-supervised domain adaptation (SSDA) is imperative when the labeled data from the source domain is statistically different from the partially labeled data from the target domain. Most prior SSDA research is centrally performed, requiring access to both source and target data. However, data in many fields nowadays is generated by distributed end devices. Due to privacy concerns, the data might be locally stored and cannot be shared, resulting in the ineffectiveness of existing SSDA research. This paper proposes an innovative approach to achieve SSDA over multiple distributed and confidential datasets, named by Federated Semi-Supervised Domain Adaptation (FSSDA). FSSDA integrates SSDA with federated learning based on strategically designed knowledge distillation techniques, whose efficiency is improved by performing source and target training in parallel. Moreover, FSSDA controls the amount of knowledge transferred across domains by properly selecting a key parameter, i.e., the imitation parameter. Further, the proposed FSSDA can be effectively generalized to multi-source domain adaptation scenarios. Extensive experiments are conducted to demonstrate the effectiveness and efficiency of FSSDA design.