Breaking BERT: Understanding its Vulnerabilities for Named Entity Recognition through Adversarial Attack
This highlights critical weaknesses in widely used NLP models for tasks like NER, which is incremental as it builds on existing adversarial attack research.
The paper investigates the vulnerability of BERT models to adversarial attacks in Named Entity Recognition, finding that 20.2-45.0% of entities are predicted completely wrong and 29.3-53.3% partially wrong due to context variations.
Both generic and domain-specific BERT models are widely used for natural language processing (NLP) tasks. In this paper we investigate the vulnerability of BERT models to variation in input data for Named Entity Recognition (NER) through adversarial attack. Experimental results show that BERT models are vulnerable to variation in the entity context with 20.2 to 45.0% of entities predicted completely wrong and another 29.3 to 53.3% of entities predicted wrong partially. BERT models seem most vulnerable to changes in the local context of entities and often a single change is sufficient to fool the model. The domain-specific BERT model trained from scratch (SciBERT) is more vulnerable than the original BERT model or the domain-specific model that retains the BERT vocabulary (BioBERT). We also find that BERT models are particularly vulnerable to emergent entities. Our results chart the vulnerabilities of BERT models for NER and emphasize the importance of further research into uncovering and reducing these weaknesses.