Federated One-Shot Learning with Data Privacy and Objective-Hiding
This addresses a critical privacy gap in federated learning for applications requiring secure collaborative learning, though it appears incremental by building on known techniques.
The paper tackles the problem of ensuring both data privacy for clients and objective privacy for the federator in federated learning, presenting a novel approach that combines knowledge distillation and private information retrieval to achieve strong information-theoretic guarantees and outperform existing methods.
Privacy in federated learning is crucial, encompassing two key aspects: safeguarding the privacy of clients' data and maintaining the privacy of the federator's objective from the clients. While the first aspect has been extensively studied, the second has received much less attention. We present a novel approach that addresses both concerns simultaneously, drawing inspiration from techniques in knowledge distillation and private information retrieval to provide strong information-theoretic privacy guarantees. Traditional private function computation methods could be used here; however, they are typically limited to linear or polynomial functions. To overcome these constraints, our approach unfolds in three stages. In stage 0, clients perform the necessary computations locally. In stage 1, these results are shared among the clients, and in stage 2, the federator retrieves its desired objective without compromising the privacy of the clients' data. The crux of the method is a carefully designed protocol that combines secret-sharing-based multi-party computation and a graph-based private information retrieval scheme. We show that our method outperforms existing tools from the literature when properly adapted to this setting.