AMR Parsing with Instruction Fine-tuned Pre-trained Language Models
This work addresses the problem of improving parsing accuracy for natural language processing researchers and practitioners, but it is incremental as it applies an existing fine-tuning method to a new task.
The paper tackled AMR parsing by fine-tuning instruction-tuned language models (FLAN-T5) on parsing tasks excluded from prior collections, achieving new state-of-the-art results with Smatch scores of 86.4 on AMR2.0, 84.9 on AMR3.0, and 82.3 on BioAMR.
Instruction fine-tuned language models on a collection of instruction annotated datasets (FLAN) have shown highly effective to improve model performance and generalization to unseen tasks. However, a majority of standard parsing tasks including abstract meaning representation (AMR), universal dependency (UD), semantic role labeling (SRL) has been excluded from the FLAN collections for both model training and evaluations. In this paper, we take one of such instruction fine-tuned pre-trained language models, i.e. FLAN-T5, and fine-tune them for AMR parsing. Our extensive experiments on various AMR parsing tasks including AMR2.0, AMR3.0 and BioAMR indicate that FLAN-T5 fine-tuned models out-perform previous state-of-the-art models across all tasks. In addition, full fine-tuning followed by the parameter efficient fine-tuning, LoRA, further improves the model performances, setting new state-of-the-arts in Smatch on AMR2.0 (86.4), AMR3.0 (84.9) and BioAMR (82.3).