Federated Transfer Learning with Differential Privacy
This work addresses privacy and heterogeneity issues in distributed data analysis for federated learning applications, but it is incremental as it builds on existing privacy models and statistical methods.
The paper tackles the challenges of data heterogeneity and privacy in federated learning by proposing a federated transfer learning framework with differential privacy, showing that federated differential privacy is an intermediate model between local and central privacy models and quantifying privacy costs for statistical problems like mean estimation and linear regression.
Federated learning has emerged as a powerful framework for analysing distributed data, yet two challenges remain pivotal: heterogeneity across sites and privacy of local data. In this paper, we address both challenges within a federated transfer learning framework, aiming to enhance learning on a target data set by leveraging information from multiple heterogeneous source data sets while adhering to privacy constraints. We rigorously formulate the notion of federated differential privacy, which offers privacy guarantees for each data set without assuming a trusted central server. Under this privacy model, we study three classical statistical problems: univariate mean estimation, low-dimensional linear regression, and high-dimensional linear regression. By investigating the minimax rates and quantifying the cost of privacy in each problem, we show that federated differential privacy is an intermediate privacy model between the well-established local and central models of differential privacy. Our analyses account for data heterogeneity and privacy, highlighting the fundamental costs associated with each factor and the benefits of knowledge transfer in federated learning.