Structured Tuning for Semantic Role Labeling
This work addresses the problem of enhancing neural SRL systems with structured knowledge for NLP researchers, representing an incremental improvement by combining neural expressiveness with constrained decoding mechanisms.
The paper tackles the challenge of integrating structured constraints into neural semantic role labeling models by proposing a structured tuning framework that applies softened constraints only during training, resulting in improved performance over a RoBERTa baseline and consistent gains in low-resource scenarios.
Recent neural network-driven semantic role labeling (SRL) systems have shown impressive improvements in F1 scores. These improvements are due to expressive input representations, which, at least at the surface, are orthogonal to knowledge-rich constrained decoding mechanisms that helped linear SRL models. Introducing the benefits of structure to inform neural models presents a methodological challenge. In this paper, we present a structured tuning framework to improve models using softened constraints only at training time. Our framework leverages the expressiveness of neural networks and provides supervision with structured loss components. We start with a strong baseline (RoBERTa) to validate the impact of our approach, and show that our framework outperforms the baseline by learning to comply with declarative constraints. Additionally, our experiments with smaller training sizes show that we can achieve consistent improvements under low-resource scenarios.