Feature-based Federated Transfer Learning: Communication Efficiency, Robustness and Privacy
This work addresses communication bottlenecks in federated learning for distributed systems, offering improvements in efficiency, robustness, and privacy, though it appears incremental as it builds on existing federated and transfer learning paradigms.
The paper tackles communication inefficiency in federated learning by proposing a feature-based federated transfer learning approach that reduces uplink payload by multiple orders of magnitude compared to existing methods, as demonstrated through experiments on image classification and NLP tasks.
In this paper, we propose feature-based federated transfer learning as a novel approach to improve communication efficiency by reducing the uplink payload by multiple orders of magnitude compared to that of existing approaches in federated learning and federated transfer learning. Specifically, in the proposed feature-based federated learning, we design the extracted features and outputs to be uploaded instead of parameter updates. For this distributed learning model, we determine the required payload and provide comparisons with the existing schemes. Subsequently, we analyze the robustness of feature-based federated transfer learning against packet loss, data insufficiency, and quantization. Finally, we address privacy considerations by defining and analyzing label privacy leakage and feature privacy leakage, and investigating mitigating approaches. For all aforementioned analyses, we evaluate the performance of the proposed learning scheme via experiments on an image classification task and a natural language processing task to demonstrate its effectiveness.