Disentangled Federated Learning for Tackling Attributes Skew via Invariant Aggregation and Diversity Transferring
This addresses performance and stability issues in federated learning for applications with skewed data distributions, though it is an incremental improvement over existing methods.
The paper tackled the problem of attributes skew in federated learning, which causes inconsistent optimization and reduced performance, by proposing disentangled federated learning (DFL) that separates domain-specific and cross-invariant attributes into two branches with alternating optimization, resulting in higher performance, better interpretability, and faster convergence compared to state-of-the-art methods.
Attributes skew hinders the current federated learning (FL) frameworks from consistent optimization directions among the clients, which inevitably leads to performance reduction and unstable convergence. The core problems lie in that: 1) Domain-specific attributes, which are non-causal and only locally valid, are indeliberately mixed into global aggregation. 2) The one-stage optimizations of entangled attributes cannot simultaneously satisfy two conflicting objectives, i.e., generalization and personalization. To cope with these, we proposed disentangled federated learning (DFL) to disentangle the domain-specific and cross-invariant attributes into two complementary branches, which are trained by the proposed alternating local-global optimization independently. Importantly, convergence analysis proves that the FL system can be stably converged even if incomplete client models participate in the global aggregation, which greatly expands the application scope of FL. Extensive experiments verify that DFL facilitates FL with higher performance, better interpretability, and faster convergence rate, compared with SOTA FL methods on both manually synthesized and realistic attributes skew datasets.