Source Anonymity for Private Random Walk Decentralized Learning
This addresses the open problem of data privacy for users in decentralized learning systems, though it is incremental as it builds on existing methods with specific graph structures.
The paper tackles the problem of preserving data privacy in random walk-based decentralized learning by proposing a privacy-preserving algorithm that uses public-key cryptography and anonymization to hide the source's identity, achieving theoretical guarantees for anonymity on random regular graphs.
This paper considers random walk-based decentralized learning, where at each iteration of the learning process, one user updates the model and sends it to a randomly chosen neighbor until a convergence criterion is met. Preserving data privacy is a central concern and open problem in decentralized learning. We propose a privacy-preserving algorithm based on public-key cryptography and anonymization. In this algorithm, the user updates the model and encrypts the result using a distant user's public key. The encrypted result is then transmitted through the network with the goal of reaching that specific user. The key idea is to hide the source's identity so that, when the destination user decrypts the result, it does not know who the source was. The challenge is to design a network-dependent probability distribution (at the source) over the potential destinations such that, from the receiver's perspective, all users have a similar likelihood of being the source. We introduce the problem and construct a scheme that provides anonymity with theoretical guarantees. We focus on random regular graphs to establish rigorous guarantees.