Heterogeneous Federated Learning with Convolutional and Spiking Neural Networks
This work addresses the problem of model heterogeneity in federated learning for edge computing platforms, but it is incremental as it benchmarks existing aggregation approaches.
The paper tackled the challenge of aggregating different types of neural networks, specifically convolutional neural networks (CNNs) and spiking neural networks (SNNs), in federated learning systems, and found that a CNN-SNN fusion framework achieved the best performance on the MNIST dataset.
Federated learning (FL) has emerged as a promising paradigm for training models on decentralized data while safeguarding data privacy. Most existing FL systems, however, assume that all machine learning models are of the same type, although it becomes more likely that different edge devices adopt different types of AI models, including both conventional analogue artificial neural networks (ANNs) and biologically more plausible spiking neural networks (SNNs). This diversity empowers the efficient handling of specific tasks and requirements, showcasing the adaptability and versatility of edge computing platforms. One main challenge of such heterogeneous FL system lies in effectively aggregating models from the local devices in a privacy-preserving manner. To address the above issue, this work benchmarks FL systems containing both convoluntional neural networks (CNNs) and SNNs by comparing various aggregation approaches, including federated CNNs, federated SNNs, federated CNNs for SNNs, federated SNNs for CNNs, and federated CNNs with SNN fusion. Experimental results demonstrate that the CNN-SNN fusion framework exhibits the best performance among the above settings on the MNIST dataset. Additionally, intriguing phenomena of competitive suppression are noted during the convergence process of multi-model FL.