A survey on secure decentralized optimization and learning
It tackles security issues in decentralized systems for large-scale decision-making and machine learning, but as a survey, it is incremental in summarizing existing work rather than introducing new methods.
This survey addresses the privacy and security risks in decentralized optimization and learning, such as data inference and model accuracy impairment by malicious agents, by providing a comprehensive tutorial on advancements in secure frameworks and algorithms over the past decade.
Decentralized optimization has become a standard paradigm for solving large-scale decision-making problems and training large machine learning models without centralizing data. However, this paradigm introduces new privacy and security risks, with malicious agents potentially able to infer private data or impair the model accuracy. Over the past decade, significant advancements have been made in developing secure decentralized optimization and learning frameworks and algorithms. This survey provides a comprehensive tutorial on these advancements. We begin with the fundamentals of decentralized optimization and learning, highlighting centralized aggregation and distributed consensus as key modules exposed to security risks in federated and distributed optimization, respectively. Next, we focus on privacy-preserving algorithms, detailing three cryptographic tools and their integration into decentralized optimization and learning systems. Additionally, we examine resilient algorithms, exploring the design and analysis of resilient aggregation and consensus protocols that support these systems. We conclude the survey by discussing current trends and potential future directions.