GLow -- A Novel, Flower-Based Simulated Gossip Learning Strategy
This work addresses the need for modular and scalable simulation tools for researchers in decentralized machine learning, though it is incremental as it extends an existing framework.
The authors tackled the challenge of simulating decentralized Gossip Learning systems by introducing GLow, a novel simulation strategy built on the Flower Framework, achieving accuracies of over 0.98 on MNIST and 0.75 on CIFAR10 while matching the performance of centralized and federated approaches.
Fully decentralized learning algorithms are still in an early stage of development. Creating modular Gossip Learning strategies is not trivial due to convergence challenges and Byzantine faults intrinsic in systems of decentralized nature. Our contribution provides a novel means to simulate custom Gossip Learning systems by leveraging the state-of-the-art Flower Framework. Specifically, we introduce GLow, which will allow researchers to train and assess scalability and convergence of devices, across custom network topologies, before making a physical deployment. The Flower Framework is selected for being a simulation featured library with a very active community on Federated Learning research. However, Flower exclusively includes vanilla Federated Learning strategies and, thus, is not originally designed to perform simulations without a centralized authority. GLow is presented to fill this gap and make simulation of Gossip Learning systems possible. Results achieved by GLow in the MNIST and CIFAR10 datasets, show accuracies over 0.98 and 0.75 respectively. More importantly, GLow performs similarly in terms of accuracy and convergence to its analogous Centralized and Federated approaches in all designed experiments.