Unveiling the Role of Randomization in Multiclass Adversarial Classification: Insights from Graph Theory
This work addresses the problem of enhancing adversarial robustness in multiclass classification for machine learning practitioners, providing theoretical insights that are incremental to existing binary classification studies.
The paper tackles the limited theoretical understanding of randomization for adversarial robustness in multiclass classification by analyzing discrete data distributions using graph theory, identifying three structural conditions necessary for robustness improvement and showing that randomization can significantly reduce optimal adversarial risk in certain distributions.
Randomization as a mean to improve the adversarial robustness of machine learning models has recently attracted significant attention. Unfortunately, much of the theoretical analysis so far has focused on binary classification, providing only limited insights into the more complex multiclass setting. In this paper, we take a step toward closing this gap by drawing inspiration from the field of graph theory. Our analysis focuses on discrete data distributions, allowing us to cast the adversarial risk minimization problems within the well-established framework of set packing problems. By doing so, we are able to identify three structural conditions on the support of the data distribution that are necessary for randomization to improve robustness. Furthermore, we are able to construct several data distributions where (contrarily to binary classification) switching from a deterministic to a randomized solution significantly reduces the optimal adversarial risk. These findings highlight the crucial role randomization can play in enhancing robustness to adversarial attacks in multiclass classification.