Research on Named Entity Recognition in Improved transformer with R-Drop structure
This is an incremental improvement for named entity recognition tasks, potentially benefiting NLP applications.
The paper tackles named entity recognition by proposing XLNet-Transformer-R, which combines XLNet, Transformer with relative positional encodings, and R-Drop to enhance generalization and accuracy. It demonstrates effectiveness through ablation experiments on MSRA and comparisons on four datasets.
To enhance the generalization ability of the model and improve the effectiveness of the transformer for named entity recognition tasks, the XLNet-Transformer-R model is proposed in this paper. The XLNet pre-trained model and the Transformer encoder with relative positional encodings are combined to enhance the model's ability to process long text and learn contextual information to improve robustness. To prevent overfitting, the R-Drop structure is used to improve the generalization capability and enhance the accuracy of the model in named entity recognition tasks. The model in this paper performs ablation experiments on the MSRA dataset and comparison experiments with other models on four datasets with excellent performance, demonstrating the strategic effectiveness of the XLNet-Transformer-R model.