Dependency Grammar Induction with a Neural Variational Transition-based Parser
This work addresses the efficiency bottleneck in unsupervised dependency parsing for NLP researchers, offering a faster alternative to graph-based models with competitive accuracy.
The paper tackled dependency grammar induction without annotated data by proposing a neural transition-based parser with variational inference, achieving performance comparable to state-of-the-art graph-based models while significantly increasing parsing speed, as shown on English Penn Treebank and Universal Dependency Treebank datasets.
Dependency grammar induction is the task of learning dependency syntax without annotated training data. Traditional graph-based models with global inference achieve state-of-the-art results on this task but they require $O(n^3)$ run time. Transition-based models enable faster inference with $O(n)$ time complexity, but their performance still lags behind. In this work, we propose a neural transition-based parser for dependency grammar induction, whose inference procedure utilizes rich neural features with $O(n)$ time complexity. We train the parser with an integration of variational inference, posterior regularization and variance reduction techniques. The resulting framework outperforms previous unsupervised transition-based dependency parsers and achieves performance comparable to graph-based models, both on the English Penn Treebank and on the Universal Dependency Treebank. In an empirical comparison, we show that our approach substantially increases parsing speed over graph-based models.