Text-CRS: A Generalized Certified Robustness Framework against Textual Adversarial Attacks
This work addresses the need for certified robustness in NLP, offering a generalized solution against multiple adversarial operations, which is a step beyond incremental improvements by extending provable guarantees to new attack types.
The paper tackles the problem of providing provable robustness guarantees for language models against textual adversarial attacks, proposing Text-CRS, a framework based on randomized smoothing that addresses four word-level operations and achieves significant accuracy improvements, including outperforming state-of-the-art methods for synonym substitution attacks.
The language models, especially the basic text classification models, have been shown to be susceptible to textual adversarial attacks such as synonym substitution and word insertion attacks. To defend against such attacks, a growing body of research has been devoted to improving the model robustness. However, providing provable robustness guarantees instead of empirical robustness is still widely unexplored. In this paper, we propose Text-CRS, a generalized certified robustness framework for natural language processing (NLP) based on randomized smoothing. To our best knowledge, existing certified schemes for NLP can only certify the robustness against $\ell_0$ perturbations in synonym substitution attacks. Representing each word-level adversarial operation (i.e., synonym substitution, word reordering, insertion, and deletion) as a combination of permutation and embedding transformation, we propose novel smoothing theorems to derive robustness bounds in both permutation and embedding space against such adversarial operations. To further improve certified accuracy and radius, we consider the numerical relationships between discrete words and select proper noise distributions for the randomized smoothing. Finally, we conduct substantial experiments on multiple language models and datasets. Text-CRS can address all four different word-level adversarial operations and achieve a significant accuracy improvement. We also provide the first benchmark on certified accuracy and radius of four word-level operations, besides outperforming the state-of-the-art certification against synonym substitution attacks.