Annotating FrameNet via Structure-Conditioned Language Generation
This work addresses the challenge of automating linguistic annotation tasks for NLP researchers, but it is incremental as it builds on existing methods with limited downstream utility.
The paper tackled the problem of generating new sentences that preserve a given semantic structure using the FrameNet formalism, and found that conditioning on explicit semantic information produced generations with high human acceptance, but the generated annotations were only effective for data augmentation in low-resource settings for frame-semantic role labeling.
Despite the remarkable generative capabilities of language models in producing naturalistic language, their effectiveness on explicit manipulation and generation of linguistic structures remain understudied. In this paper, we investigate the task of generating new sentences preserving a given semantic structure, following the FrameNet formalism. We propose a framework to produce novel frame-semantically annotated sentences following an overgenerate-and-filter approach. Our results show that conditioning on rich, explicit semantic information tends to produce generations with high human acceptance, under both prompting and finetuning. Our generated frame-semantic structured annotations are effective at training data augmentation for frame-semantic role labeling in low-resource settings; however, we do not see benefits under higher resource settings. Our study concludes that while generating high-quality, semantically rich data might be within reach, the downstream utility of such generations remains to be seen, highlighting the outstanding challenges with automating linguistic annotation tasks.