Multi-Task Cross-Lingual Sequence Tagging from Scratch
This work addresses sequence tagging for NLP researchers and practitioners by offering a versatile model that improves performance through multi-task and cross-lingual training, though it is incremental as it builds on existing neural architectures.
The authors tackled the problem of sequence tagging across multiple languages and tasks by developing a deep hierarchical recurrent neural network that is task- and language-independent, achieving state-of-the-art results on benchmarks like POS tagging, chunking, and NER.
We present a deep hierarchical recurrent neural network for sequence tagging. Given a sequence of words, our model employs deep gated recurrent units on both character and word levels to encode morphology and context information, and applies a conditional random field layer to predict the tags. Our model is task independent, language independent, and feature engineering free. We further extend our model to multi-task and cross-lingual joint training by sharing the architecture and parameters. Our model achieves state-of-the-art results in multiple languages on several benchmark tasks including POS tagging, chunking, and NER. We also demonstrate that multi-task and cross-lingual joint training can improve the performance in various cases.