On the effects of similarity metrics in decentralized deep learning under distributional shift
This work addresses a pressing issue for organizations or users seeking privacy-preserving collaboration in decentralized learning, but it appears incremental as it focuses on empirical analysis of existing metrics.
The paper tackled the challenge of identifying compatible collaborators in decentralized deep learning under data heterogeneity by investigating the effectiveness of various similarity metrics for model merging, finding insights into their performance across datasets with distribution shifts.
Decentralized Learning (DL) enables privacy-preserving collaboration among organizations or users to enhance the performance of local deep learning models. However, model aggregation becomes challenging when client data is heterogeneous, and identifying compatible collaborators without direct data exchange remains a pressing issue. In this paper, we investigate the effectiveness of various similarity metrics in DL for identifying peers for model merging, conducting an empirical analysis across multiple datasets with distribution shifts. Our research provides insights into the performance of these metrics, examining their role in facilitating effective collaboration. By exploring the strengths and limitations of these metrics, we contribute to the development of robust DL methods.