PaDPaF: Partial Disentanglement with Partially-Federated GANs
This work addresses the need for personalized generative models in heterogeneous federated settings, which is incremental as it builds on existing federated learning techniques.
The paper tackles the problem of learning personalized generative models in federated learning by proposing an architecture that combines global and local models, achieving privacy and personalization through implicit disentanglement of content and style, with results showing high accuracy in label prediction and enabling applications like data anonymization.
Federated learning has become a popular machine learning paradigm with many potential real-life applications, including recommendation systems, the Internet of Things (IoT), healthcare, and self-driving cars. Though most current applications focus on classification-based tasks, learning personalized generative models remains largely unexplored, and their benefits in the heterogeneous setting still need to be better understood. This work proposes a novel architecture combining global client-agnostic and local client-specific generative models. We show that using standard techniques for training federated models, our proposed model achieves privacy and personalization by implicitly disentangling the globally consistent representation (i.e. content) from the client-dependent variations (i.e. style). Using such decomposition, personalized models can generate locally unseen labels while preserving the given style of the client and can predict the labels for all clients with high accuracy by training a simple linear classifier on the global content features. Furthermore, disentanglement enables other essential applications, such as data anonymization, by sharing only the content. Extensive experimental evaluation corroborates our findings, and we also discuss a theoretical motivation for the proposed approach.