Evaluating the Utility of Hand-crafted Features in Sequence Labelling
This addresses the practical problem of optimizing neural models for NLP researchers and practitioners, though it is incremental in showing feature utility.
The paper tackles the problem of whether hand-crafted features are useful for deep learning in sequence labeling, specifically named entity recognition, and finds that incorporating them via a hybrid approach with an auto-encoder loss improves F1 score to 91.89 on CoNLL-2003 and reduces training requirements by 40%.
Conventional wisdom is that hand-crafted features are redundant for deep learning models, as they already learn adequate representations of text automatically from corpora. In this work, we test this claim by proposing a new method for exploiting handcrafted features as part of a novel hybrid learning approach, incorporating a feature auto-encoder loss component. We evaluate on the task of named entity recognition (NER), where we show that including manual features for part-of-speech, word shapes and gazetteers can improve the performance of a neural CRF model. We obtain a $F_1$ of 91.89 for the CoNLL-2003 English shared task, which significantly outperforms a collection of highly competitive baseline models. We also present an ablation study showing the importance of auto-encoding, over using features as either inputs or outputs alone, and moreover, show including the autoencoder components reduces training requirements to 60\%, while retaining the same predictive accuracy.