Robustness Tests of NLP Machine Learning Models: Search and Semantically Replace
This addresses the need for robust NLP models in applications like text classification or sentiment analysis, but it is incremental as it builds on existing robustness testing methods.
The paper tackles the problem of assessing robustness in NLP machine learning models by proposing a Search and Semantically Replace strategy, which identifies important text parts and replaces them with semantically similar words, and provides an empirical comparison showing robustness performance across three model types with different text representations.
This paper proposes a strategy to assess the robustness of different machine learning models that involve natural language processing (NLP). The overall approach relies upon a Search and Semantically Replace strategy that consists of two steps: (1) Search, which identifies important parts in the text; (2) Semantically Replace, which finds replacements for the important parts, and constrains the replaced tokens with semantically similar words. We introduce different types of Search and Semantically Replace methods designed specifically for particular types of machine learning models. We also investigate the effectiveness of this strategy and provide a general framework to assess a variety of machine learning models. Finally, an empirical comparison is provided of robustness performance among three different model types, each with a different text representation.