MAPL: Model Agnostic Peer-to-peer Learning
This addresses the challenge of decentralized learning for heterogeneous clients, offering a novel solution that is incremental in improving collaboration methods.
The paper tackles the problem of effective collaboration among heterogeneous clients in decentralized settings by introducing MAPL, a model-agnostic peer-to-peer learning approach that simultaneously learns personalized models and a collaboration graph, achieving competitive or superior performance compared to centralized counterparts without a central server.
Effective collaboration among heterogeneous clients in a decentralized setting is a rather unexplored avenue in the literature. To structurally address this, we introduce Model Agnostic Peer-to-peer Learning (coined as MAPL) a novel approach to simultaneously learn heterogeneous personalized models as well as a collaboration graph through peer-to-peer communication among neighboring clients. MAPL is comprised of two main modules: (i) local-level Personalized Model Learning (PML), leveraging a combination of intra- and inter-client contrastive losses; (ii) network-wide decentralized Collaborative Graph Learning (CGL) dynamically refining collaboration weights in a privacy-preserving manner based on local task similarities. Our extensive experimentation demonstrates the efficacy of MAPL and its competitive (or, in most cases, superior) performance compared to its centralized model-agnostic counterparts, without relying on any central server. Our code is available and can be accessed here: https://github.com/SayakMukherjee/MAPL