Transition-based Parsing with Lighter Feed-Forward Networks
This work addresses efficiency improvements for dependency parsing across multiple languages, but it is incremental as it builds on existing transition-based parsers with feed-forward networks.
The paper tackled the problem of building lighter dependency parsers by reducing feature sets and embedding sizes, showing that grand-daughter features can be removed for most languages without significant accuracy loss and embeddings can be notably reduced.
We explore whether it is possible to build lighter parsers, that are statistically equivalent to their corresponding standard version, for a wide set of languages showing different structures and morphologies. As testbed, we use the Universal Dependencies and transition-based dependency parsers trained on feed-forward networks. For these, most existing research assumes de facto standard embedded features and relies on pre-computation tricks to obtain speed-ups. We explore how these features and their size can be reduced and whether this translates into speed-ups with a negligible impact on accuracy. The experiments show that grand-daughter features can be removed for the majority of treebanks without a significant (negative or positive) LAS difference. They also show how the size of the embeddings can be notably reduced.