Segmental Recurrent Neural Networks
This work addresses sequence labeling tasks in natural language processing and handwriting recognition by improving accuracy through explicit segment representation, representing an incremental advance over existing methods.
The paper tackles the problem of sequence labeling with explicit segment modeling by introducing segmental recurrent neural networks (SRNNs), which jointly model segmentations and labelings, and reports that SRNNs achieve substantially higher accuracies compared to methods like BIO tagging and connectionist temporal classification in experiments on handwriting recognition and Chinese word segmentation/POS tagging.
We introduce segmental recurrent neural networks (SRNNs) which define, given an input sequence, a joint probability distribution over segmentations of the input and labelings of the segments. Representations of the input segments (i.e., contiguous subsequences of the input) are computed by encoding their constituent tokens using bidirectional recurrent neural nets, and these "segment embeddings" are used to define compatibility scores with output labels. These local compatibility scores are integrated using a global semi-Markov conditional random field. Both fully supervised training -- in which segment boundaries and labels are observed -- as well as partially supervised training -- in which segment boundaries are latent -- are straightforward. Experiments on handwriting recognition and joint Chinese word segmentation/POS tagging show that, compared to models that do not explicitly represent segments such as BIO tagging schemes and connectionist temporal classification (CTC), SRNNs obtain substantially higher accuracies.