FedSecurity: Benchmarking Attacks and Defenses in Federated Learning and Federated LLMs
This provides a standardized tool for researchers and practitioners in federated learning to test security measures, though it is incremental as it builds on existing libraries and methods.
The paper tackles the problem of evaluating adversarial attacks and defenses in federated learning by introducing FedSecurity, a benchmark integrated into the FedML library, which simplifies simulation and customization across various models and datasets, including demonstrations with large language models.
This paper introduces FedSecurity, an end-to-end benchmark that serves as a supplementary component of the FedML library for simulating adversarial attacks and corresponding defense mechanisms in Federated Learning (FL). FedSecurity eliminates the need for implementing the fundamental FL procedures, e.g., FL training and data loading, from scratch, thus enables users to focus on developing their own attack and defense strategies. It contains two key components, including FedAttacker that conducts a variety of attacks during FL training, and FedDefender that implements defensive mechanisms to counteract these attacks. FedSecurity has the following features: i) It offers extensive customization options to accommodate a broad range of machine learning models (e.g., Logistic Regression, ResNet, and GAN) and FL optimizers (e.g., FedAVG, FedOPT, and FedNOVA); ii) it enables exploring the effectiveness of attacks and defenses across different datasets and models; and iii) it supports flexible configuration and customization through a configuration file and some APIs. We further demonstrate FedSecurity's utility and adaptability through federated training of Large Language Models (LLMs) to showcase its potential on a wide range of complex applications.