Hierarchically-Refined Label Attention Network for Sequence Labeling
This work addresses sequence labeling tasks like POS tagging, NER, and CCG supertagging, offering incremental improvements in efficiency and performance.
The paper tackled the problem of neural sequence labeling by proposing a hierarchically-refined label attention network to better represent label sequences and capture long-term dependencies, resulting in improved tagging accuracy and significantly faster training and testing compared to BiLSTM-CRF.
CRF has been used as a powerful model for statistical sequence labeling. For neural sequence labeling, however, BiLSTM-CRF does not always lead to better results compared with BiLSTM-softmax local classification. This can be because the simple Markov label transition model of CRF does not give much information gain over strong neural encoding. For better representing label sequences, we investigate a hierarchically-refined label attention network, which explicitly leverages label embeddings and captures potential long-term label dependency by giving each word incrementally refined label distributions with hierarchical attention. Results on POS tagging, NER and CCG supertagging show that the proposed model not only improves the overall tagging accuracy with similar number of parameters, but also significantly speeds up the training and testing compared to BiLSTM-CRF.