Hybrid Neural Models For Sequence Modelling: The Best Of Three Worlds
This work addresses sequence labeling tasks, but it is incremental as it combines existing methods without introducing a fundamentally new approach.
The authors tackled sequence labeling by combining bidirectional RNNs, encoder-decoder, and Transformer models into a hybrid neural architecture, achieving results close to state-of-the-art on three tasks and outperforming it in some cases.
We propose a neural architecture with the main characteristics of the most successful neural models of the last years: bidirectional RNNs, encoder-decoder, and the Transformer model. Evaluation on three sequence labelling tasks yields results that are close to the state-of-the-art for all tasks and better than it for some of them, showing the pertinence of this hybrid architecture for this kind of tasks.