Learning Dynamic Feature Selection for Fast Sequential Prediction
This work addresses efficiency bottlenecks for NLP practitioners by enabling faster sequential prediction in tasks like tagging and parsing, though it is incremental as it builds on cascade methods.
The paper tackles the problem of reducing computation and increasing speed in NLP classifiers by partitioning features into ordered templates to achieve high confidence with fewer features, resulting in over a five-fold reduction in run-time for POS tagging and parsing while maintaining accuracy above 97% and 88.5% LAS, respectively, and more than 2x speed increase for NER with F1 above 88.
We present paired learning and inference algorithms for significantly reducing computation and increasing speed of the vector dot products in the classifiers that are at the heart of many NLP components. This is accomplished by partitioning the features into a sequence of templates which are ordered such that high confidence can often be reached using only a small fraction of all features. Parameter estimation is arranged to maximize accuracy and early confidence in this sequence. Our approach is simpler and better suited to NLP than other related cascade methods. We present experiments in left-to-right part-of-speech tagging, named entity recognition, and transition-based dependency parsing. On the typical benchmarking datasets we can preserve POS tagging accuracy above 97% and parsing LAS above 88.5% both with over a five-fold reduction in run-time, and NER F1 above 88 with more than 2x increase in speed.