Lex-BERT: Enhancing BERT based NER with lexicons
This work provides an incremental improvement for Chinese NER, specifically for researchers and practitioners working with BERT-based models.
This paper introduces Lex-BERT, a method for Chinese Named Entity Recognition (NER) that integrates lexicon information into BERT by marking word boundaries with special tokens. This approach allows direct encoding by BERT without new parameters or word embeddings, achieving superior performance compared to FLAT and other baselines on Ontonotes and ZhCrossNER datasets.
In this work, we represent Lex-BERT, which incorporates the lexicon information into Chinese BERT for named entity recognition (NER) tasks in a natural manner. Instead of using word embeddings and a newly designed transformer layer as in FLAT, we identify the boundary of words in the sentences using special tokens, and the modified sentence will be encoded directly by BERT. Our model does not introduce any new parameters and are more efficient than FLAT. In addition, we do not require any word embeddings accompanying the lexicon collection. Experiments on Ontonotes and ZhCrossNER show that our model outperforms FLAT and other baselines.