ALIGN-FL: Architecture-independent Learning through Invariant Generative component sharing in Federated Learning
This addresses privacy and data heterogeneity issues in federated learning for cross-silo collaborations, but it is incremental as it builds on existing methods like DP-SGD and VAEs.
The paper tackles the challenge of learning from highly disjoint data distributions in federated learning by proposing ALIGN-FL, which uses selective sharing of generative components to enable privacy-preserving learning, achieving effective mapping of sensitive outliers to typical data points while maintaining utility in extreme Non-IID scenarios.
We present ALIGN-FL, a novel approach to distributed learning that addresses the challenge of learning from highly disjoint data distributions through selective sharing of generative components. Instead of exchanging full model parameters, our framework enables privacy-preserving learning by transferring only generative capabilities across clients, while the server performs global training using synthetic samples. Through complementary privacy mechanisms: DP-SGD with adaptive clipping and Lipschitz regularized VAE decoders and a stateful architecture supporting heterogeneous clients, we experimentally validate our approach on MNIST and Fashion-MNIST datasets with cross-domain outliers. Our analysis demonstrates that both privacy mechanisms effectively map sensitive outliers to typical data points while maintaining utility in extreme Non-IID scenarios typical of cross-silo collaborations. Index Terms: Client-invariant Learning, Federated Learning (FL), Privacy-preserving Generative Models, Non-Independent and Identically Distributed (Non-IID), Heterogeneous Architectures