Attack on Unfair ToS Clause Detection: A Case Study using Universal Adversarial Triggers
This highlights a security vulnerability in consumer protection tools, which is incremental as it applies known adversarial methods to a new domain.
The paper demonstrates that transformer-based systems for detecting unfair clauses in Terms of Service agreements are vulnerable to adversarial attacks using universal triggers, with minor text perturbations significantly reducing detection performance, and human evaluation shows these triggers are hard to detect.
Recent work has demonstrated that natural language processing techniques can support consumer protection by automatically detecting unfair clauses in the Terms of Service (ToS) Agreement. This work demonstrates that transformer-based ToS analysis systems are vulnerable to adversarial attacks. We conduct experiments attacking an unfair-clause detector with universal adversarial triggers. Experiments show that a minor perturbation of the text can considerably reduce the detection performance. Moreover, to measure the detectability of the triggers, we conduct a detailed human evaluation study by collecting both answer accuracy and response time from the participants. The results show that the naturalness of the triggers remains key to tricking readers.