Single-Round Scalable Analytic Federated Learning
This provides a scalable and efficient solution for federated vision tasks, addressing key challenges like communication overhead and heterogeneous data, though it is incremental in improving upon existing analytic FL methods.
The paper tackled the trade-off between single-round efficiency and non-linear expressivity in analytic federated learning, proposing SAFLe, which achieves state-of-the-art accuracy by using a structured head and sparse embeddings that are mathematically equivalent to linear regression, enabling single-shot aggregation.
Federated Learning (FL) is plagued by two key challenges: high communication overhead and performance collapse on heterogeneous (non-IID) data. Analytic FL (AFL) provides a single-round, data distribution invariant solution, but is limited to linear models. Subsequent non-linear approaches, like DeepAFL, regain accuracy but sacrifice the single-round benefit. In this work, we break this trade-off. We propose SAFLe, a framework that achieves scalable non-linear expressivity by introducing a structured head of bucketed features and sparse, grouped embeddings. We prove this non-linear architecture is mathematically equivalent to a high-dimensional linear regression. This key equivalence allows SAFLe to be solved with AFL's single-shot, invariant aggregation law. Empirically, SAFLe establishes a new state-of-the-art for analytic FL, significantly outperforming both linear AFL and multi-round DeepAFL in accuracy across all benchmarks, demonstrating a highly efficient and scalable solution for federated vision.