The poison of dimensionality
This reveals a fundamental security limitation for practitioners deploying large models, showing poisoning attacks can bypass state-of-the-art defenses in high-dimensional settings.
The paper demonstrates that machine learning models with sufficiently many parameters relative to honest and poisoned data points can be arbitrarily manipulated by attackers, despite robust defenses, proving a linear/logistic regression vulnerability threshold of D ≥ 169H²/P² and showing a tradeoff between model expressivity and attack surface.
This paper advances the understanding of how the size of a machine learning model affects its vulnerability to poisoning, despite state-of-the-art defenses. Given isotropic random honest feature vectors and the geometric median (or clipped mean) as the robust gradient aggregator rule, we essentially prove that, perhaps surprisingly, linear and logistic regressions with $D \geq 169 H^2/P^2$ parameters are subject to arbitrary model manipulation by poisoners, where $H$ and $P$ are the numbers of honestly labeled and poisoned data points used for training. Our experiments go on exposing a fundamental tradeoff between augmenting model expressivity and increasing the poisoners' attack surface, on both synthetic data, and on MNIST & FashionMNIST data for linear classifiers with random features. We also discuss potential implications for source-based learning and neural nets.