Neural Networks, Inside Out: Solving for Inputs Given Parameters (A Preliminary Investigation)
This addresses a security vulnerability in multi-party computation or federated learning settings where parameter updates might be leaked, posing a privacy risk for sensitive data.
The paper tackles the problem of recovering the original training dataset from observed neural network parameter updates, demonstrating that an adversary can reconstruct the dataset by solving a system of equations derived from parameter evolution, with experiments showing recovery of up to 90% of data points in simple cases.
Artificial neural network (ANN) is a supervised learning algorithm, where parameters are learned by several back-and-forth iterations of passing the inputs through the network, comparing the output with the expected labels, and correcting the parameters. Inspired by a recent work of Boer and Kramer (2020), we investigate a different problem: Suppose an observer can view how the ANN parameters evolve over many iterations, but the dataset is oblivious to him. For instance, this can be an adversary eavesdropping on a multi-party computation of an ANN parameters (where intermediate parameters are leaked). Can he form a system of equations, and solve it to recover the dataset?