Private Learning on Networks
This addresses privacy concerns for distributed machine learning systems where data is stored across multiple machines, though it appears incremental as it builds on existing secure multi-party computation ideas.
The paper tackles the privacy challenge in distributed machine learning by proposing a secure multi-party computation inspired algorithm for optimizing convex functions with non-convex components, showing it accurately learns the model and preserves privacy under network connectivity conditions.
Continual data collection and widespread deployment of machine learning algorithms, particularly the distributed variants, have raised new privacy challenges. In a distributed machine learning scenario, the dataset is stored among several machines and they solve a distributed optimization problem to collectively learn the underlying model. We present a secure multi-party computation inspired privacy preserving distributed algorithm for optimizing a convex function consisting of several possibly non-convex functions. Each individual objective function is privately stored with an agent while the agents communicate model parameters with neighbor machines connected in a network. We show that our algorithm can correctly optimize the overall objective function and learn the underlying model accurately. We further prove that under a vertex connectivity condition on the topology, our algorithm preserves privacy of individual objective functions. We establish limits on the what a coalition of adversaries can learn by observing the messages and states shared over a network.