Masked Conditional Random Fields for Sequence Labeling
This addresses a specific issue in sequence labeling for NLP tasks, but it is incremental as it modifies an existing method.
The paper tackles the problem of CRF-based models generating illegal tag sequences in sequence labeling by proposing Masked Conditional Random Field (MCRF), which imposes restrictions during training and decoding, resulting in consistent improvements with near zero additional cost.
Conditional Random Field (CRF) based neural models are among the most performant methods for solving sequence labeling problems. Despite its great success, CRF has the shortcoming of occasionally generating illegal sequences of tags, e.g. sequences containing an "I-" tag immediately after an "O" tag, which is forbidden by the underlying BIO tagging scheme. In this work, we propose Masked Conditional Random Field (MCRF), an easy to implement variant of CRF that impose restrictions on candidate paths during both training and decoding phases. We show that the proposed method thoroughly resolves this issue and brings consistent improvement over existing CRF-based models with near zero additional cost.