Back in Black: A Comparative Evaluation of Recent State-Of-The-Art Black-Box Attacks
This is an incremental contribution that helps researchers in adversarial machine learning by organizing and comparing existing methods.
The paper tackles the problem of unorganized growth in adversarial machine learning literature by systematizing recent black-box attacks since 2019, summarizing 20 attacks into four categories and providing a mathematical framework for fair comparison.
The field of adversarial machine learning has experienced a near exponential growth in the amount of papers being produced since 2018. This massive information output has yet to be properly processed and categorized. In this paper, we seek to help alleviate this problem by systematizing the recent advances in adversarial machine learning black-box attacks since 2019. Our survey summarizes and categorizes 20 recent black-box attacks. We also present a new analysis for understanding the attack success rate with respect to the adversarial model used in each paper. Overall, our paper surveys a wide body of literature to highlight recent attack developments and organizes them into four attack categories: score based attacks, decision based attacks, transfer attacks and non-traditional attacks. Further, we provide a new mathematical framework to show exactly how attack results can fairly be compared.