Training with Exploration Improves a Greedy Stack-LSTM Parser
This work addresses parsing accuracy improvements for NLP researchers, but it is incremental as it builds on existing methods.
The authors tackled the problem of improving dependency parsing accuracy by adapting a greedy Stack-LSTM parser to use training-with-exploration with dynamic oracles instead of cross-entropy minimization, resulting in very strong parsing accuracies for English and Chinese.
We adapt the greedy Stack-LSTM dependency parser of Dyer et al. (2015) to support a training-with-exploration procedure using dynamic oracles(Goldberg and Nivre, 2013) instead of cross-entropy minimization. This form of training, which accounts for model predictions at training time rather than assuming an error-free action history, improves parsing accuracies for both English and Chinese, obtaining very strong results for both languages. We discuss some modifications needed in order to get training with exploration to work well for a probabilistic neural-network.