Overparametrization improves robustness against adversarial attacks: A replication study
This is an incremental study that replicates and extends previous findings on robustness in machine learning models, relevant for researchers in adversarial machine learning.
The study empirically investigates how overparametrization affects model robustness against adversarial attacks, replicating prior work and supporting the 'universal law of robustness' with findings that overparametrization improves robustness but is insufficient for full robustness.
Overparametrization has become a de facto standard in machine learning. Despite numerous efforts, our understanding of how and where overparametrization helps model accuracy and robustness is still limited. To this end, here we conduct an empirical investigation to systemically study and replicate previous findings in this area, in particular the study by Madry et al. Together with this study, our findings support the "universal law of robustness" recently proposed by Bubeck et al. We argue that while critical for robust perception, overparametrization may not be enough to achieve full robustness and smarter architectures e.g. the ones implemented by the human visual cortex) seem inevitable.