Self-attention-based BiGRU and capsule network for named entity recognition
This work addresses named entity recognition for natural language processing, but it is incremental as it builds on existing methods like BERT and BiGRU.
The authors tackled the problem of weak character representation and insufficient sequence information capture in named entity recognition by proposing a model combining self-attention-based BiGRU and capsule networks, achieving better performance without external dictionaries on two datasets.
Named entity recognition(NER) is one of the tasks of natural language processing(NLP). In view of the problem that the traditional character representation ability is weak and the neural network method is unable to capture the important sequence information. An self-attention-based bidirectional gated recurrent unit(BiGRU) and capsule network(CapsNet) for NER is proposed. This model generates character vectors through bidirectional encoder representation of transformers(BERT) pre-trained model. BiGRU is used to capture sequence context features, and self-attention mechanism is proposed to give different focus on the information captured by hidden layer of BiGRU. Finally, we propose to use CapsNet for entity recognition. We evaluated the recognition performance of the model on two datasets. Experimental results show that the model has better performance without relying on external dictionary information.