Consensus learning: A novel decentralised ensemble learning paradigm
This work addresses the problem of secure and private distributed learning for users in decentralized systems, offering a novel paradigm that is not incremental but introduces a new approach.
The paper tackles the need for efficient and scalable distributed machine learning by introducing consensus learning, a decentralized ensemble method that combines ensemble techniques with peer-to-peer consensus protocols to ensure data privacy and Byzantine robustness, achieving performance comparable to centralized ensemble algorithms in simulations.
The widespread adoption of large-scale machine learning models in recent years highlights the need for distributed computing for efficiency and scalability. This work introduces a novel distributed machine learning paradigm -- \emph{consensus learning} -- which combines classical ensemble methods with consensus protocols deployed in peer-to-peer systems. These algorithms consist of two phases: first, participants develop their models and submit predictions for any new data inputs; second, the individual predictions are used as inputs for a communication phase, which is governed by a consensus protocol. Consensus learning ensures user data privacy, while also inheriting the safety measures against Byzantine attacks from the underlying consensus mechanism. We provide a detailed theoretical analysis for a particular consensus protocol and compare the performance of the consensus learning ensemble with centralised ensemble learning algorithms. The discussion is supplemented by various numerical simulations, which describe the robustness of the algorithms against Byzantine participants.