Fall of Giants: How popular text-based MLaaS fall against a simple evasion attack
This exposes a critical security flaw in widely used commercial ML services, which could lead to misunderstandings or misuse, though it is incremental as it focuses on a specific pipeline stage rather than a new paradigm.
The paper tackles the vulnerability of popular text-based Machine-Learning-as-a-Service (MLaaS) platforms to a simple evasion attack that injects non-readable characters to disrupt the indexing stage, demonstrating that 11 out of 12 services from major providers like Amazon and Google were deceived in a case study on hateful tweets.
The increased demand for machine learning applications made companies offer Machine-Learning-as-a-Service (MLaaS). In MLaaS (a market estimated 8000M USD by 2025), users pay for well-performing ML models without dealing with the complicated training procedure. Among MLaaS, text-based applications are the most popular ones (e.g., language translators). Given this popularity, MLaaS must provide resiliency to adversarial manipulations. For example, a wrong translation might lead to a misunderstanding between two parties. In the text domain, state-of-the-art attacks mainly focus on strategies that leverage ML models' weaknesses. Unfortunately, not much attention has been given to the other pipeline' stages, such as the indexing stage (i.e., when a sentence is converted from a textual to a numerical representation) that, if manipulated, can significantly affect the final performance of the application. In this paper, we propose a novel text evasion technique called "\textit{Zero-Width} attack" (ZeW) that leverages the injection of human non-readable characters, affecting indexing stage mechanisms. We demonstrate that our simple yet effective attack deceives MLaaS of "giants" such as Amazon, Google, IBM, and Microsoft. Our case study, based on the manipulation of hateful tweets, shows that out of 12 analyzed services, only one is resistant to our injection strategy. We finally introduce and test a simple \textit{input validation} defense that can prevent our proposed attack.