MockingBERT: A Method for Retroactively Adding Resilience to NLP Models
This addresses the need for robust NLP models against adversarial and accidental misspellings, offering a retroactive solution that is incremental in improving existing methods.
The paper tackles the problem of protecting NLP models against misspellings without compromising accuracy or requiring full re-training, achieving resilience with minimal performance loss on clean inputs.
Protecting NLP models against misspellings whether accidental or adversarial has been the object of research interest for the past few years. Existing remediations have typically either compromised accuracy or required full model re-training with each new class of attacks. We propose a novel method of retroactively adding resilience to misspellings to transformer-based NLP models. This robustness can be achieved without the need for re-training of the original NLP model and with only a minimal loss of language understanding performance on inputs without misspellings. Additionally we propose a new efficient approximate method of generating adversarial misspellings, which significantly reduces the cost needed to evaluate a model's resilience to adversarial attacks.