Stack-propagation: Improved Representation Learning for Syntax
This work addresses syntax parsing and tagging for natural language processing, offering incremental improvements over existing methods.
The paper tackled the problem of improving syntax models by using part-of-speech tags as a regularizer for learned representations, resulting in a method called stack-propagation that achieved 1.3% higher accuracy than a state-of-the-art graph-based approach and 2.7% higher than a greedy model on 19 languages from Universal Dependencies.
Traditional syntax models typically leverage part-of-speech (POS) information by constructing features from hand-tuned templates. We demonstrate that a better approach is to utilize POS tags as a regularizer of learned representations. We propose a simple method for learning a stacked pipeline of models which we call "stack-propagation". We apply this to dependency parsing and tagging, where we use the hidden layer of the tagger network as a representation of the input tokens for the parser. At test time, our parser does not require predicted POS tags. On 19 languages from the Universal Dependencies, our method is 1.3% (absolute) more accurate than a state-of-the-art graph-based approach and 2.7% more accurate than the most comparable greedy model.