Structure-aware Fine-tuning of Sequence-to-sequence Transformers for Transition-based AMR Parsing
This work addresses the need for graph well-formedness guarantees and alignments in AMR parsing for natural language processing applications, representing an incremental improvement over existing methods.
The authors tackled the problem of AMR parsing by integrating pre-trained sequence-to-sequence Transformers with a structure-aware transition-based approach, achieving new state-of-the-art results on AMR 2.0 without requiring graph re-categorization.
Predicting linearized Abstract Meaning Representation (AMR) graphs using pre-trained sequence-to-sequence Transformer models has recently led to large improvements on AMR parsing benchmarks. These parsers are simple and avoid explicit modeling of structure but lack desirable properties such as graph well-formedness guarantees or built-in graph-sentence alignments. In this work we explore the integration of general pre-trained sequence-to-sequence language models and a structure-aware transition-based approach. We depart from a pointer-based transition system and propose a simplified transition set, designed to better exploit pre-trained language models for structured fine-tuning. We also explore modeling the parser state within the pre-trained encoder-decoder architecture and different vocabulary strategies for the same purpose. We provide a detailed comparison with recent progress in AMR parsing and show that the proposed parser retains the desirable properties of previous transition-based approaches, while being simpler and reaching the new parsing state of the art for AMR 2.0, without the need for graph re-categorization.