Distilling Causal Effect from Miscellaneous Other-Class for Continual Named Entity Recognition
This addresses the problem of forgetting old entity types in continual NER for NLP applications, with incremental improvements through a novel causal approach.
The paper tackles catastrophic forgetting in continual learning for named entity recognition by identifying that the Other-Class contains old entity types, and proposes a causal framework to retrieve causality from both new types and Other-Class, resulting in outperforming state-of-the-art methods by a large margin on three benchmark datasets.
Continual Learning for Named Entity Recognition (CL-NER) aims to learn a growing number of entity types over time from a stream of data. However, simply learning Other-Class in the same way as new entity types amplifies the catastrophic forgetting and leads to a substantial performance drop. The main cause behind this is that Other-Class samples usually contain old entity types, and the old knowledge in these Other-Class samples is not preserved properly. Thanks to the causal inference, we identify that the forgetting is caused by the missing causal effect from the old data. To this end, we propose a unified causal framework to retrieve the causality from both new entity types and Other-Class. Furthermore, we apply curriculum learning to mitigate the impact of label noise and introduce a self-adaptive weight for balancing the causal effects between new entity types and Other-Class. Experimental results on three benchmark datasets show that our method outperforms the state-of-the-art method by a large margin. Moreover, our method can be combined with the existing state-of-the-art methods to improve the performance in CL-NER