Training for Fast Sequential Prediction Using Dynamic Feature Selection
This addresses efficiency bottlenecks for NLP practitioners using sequential prediction tasks, though it appears incremental as it optimizes existing classifier architectures rather than introducing a new paradigm.
The paper tackles the problem of high computational cost in NLP classifiers by developing paired learning and inference algorithms that use dynamic feature selection to reduce vector dot product calculations. The result is a system that maintains part-of-speech tagging accuracy above 97% on WSJ data while achieving over a five-fold reduction in run-time.
We present paired learning and inference algorithms for significantly reducing computation and increasing speed of the vector dot products in the classifiers that are at the heart of many NLP components. This is accomplished by partitioning the features into a sequence of templates which are ordered such that high confidence can often be reached using only a small fraction of all features. Parameter estimation is arranged to maximize accuracy and early confidence in this sequence. We present experiments in left-to-right part-of-speech tagging on WSJ, demonstrating that we can preserve accuracy above 97% with over a five-fold reduction in run-time.