Federated Learning Optimization: A Comparative Study of Data and Model Exchange Strategies in Dynamic Networks
This work addresses the efficiency of federated learning for applications in dynamic network environments, but it is incremental as it compares existing strategies without introducing a new method.
The study tackled the problem of optimizing federated learning in dynamic networks by comparing strategies for exchanging data versus model updates, finding that the efficiency of time-limited knowledge transfer can vary by up to 9.08% across different scenarios.
The promise and proliferation of large-scale dynamic federated learning gives rise to a prominent open question - is it prudent to share data or model across nodes, if efficiency of transmission and fast knowledge transfer are the prime objectives. This work investigates exactly that. Specifically, we study the choices of exchanging raw data, synthetic data, or (partial) model updates among devices. The implications of these strategies in the context of foundational models are also examined in detail. Accordingly, we obtain key insights about optimal data and model exchange mechanisms considering various environments with different data distributions and dynamic device and network connections. Across various scenarios that we considered, time-limited knowledge transfer efficiency can differ by up to 9.08\%, thus highlighting the importance of this work.