Variational Sequential Labelers for Semi-Supervised Learning
This work addresses the problem of improving sequence labeling tasks, such as named entity recognition or part-of-speech tagging, for NLP researchers and practitioners, offering incremental advances through novel model integration.
The paper tackles semi-supervised sequence labeling by introducing a family of multitask variational methods that combine a latent-variable generative model with a discriminative labeler, resulting in consistent performance improvements over standard baselines on 8 datasets, with further gains from unlabeled data.
We introduce a family of multitask variational methods for semi-supervised sequence labeling. Our model family consists of a latent-variable generative model and a discriminative labeler. The generative models use latent variables to define the conditional probability of a word given its context, drawing inspiration from word prediction objectives commonly used in learning word embeddings. The labeler helps inject discriminative information into the latent space. We explore several latent variable configurations, including ones with hierarchical structure, which enables the model to account for both label-specific and word-specific information. Our models consistently outperform standard sequential baselines on 8 sequence labeling datasets, and improve further with unlabeled data.