DocAMR: Multi-Sentence AMR Representation and Evaluation
This addresses the problem of document-level AMR parsing for NLP researchers, offering incremental improvements in representation and evaluation.
The paper tackles the lack of well-defined representation and evaluation for full-document AMR parsing by introducing a simple algorithm for unified graph representation and improving the Smatch metric for document-level comparison, re-evaluating a parser and providing a pipeline baseline.
Despite extensive research on parsing of English sentences into Abstraction Meaning Representation (AMR) graphs, which are compared to gold graphs via the Smatch metric, full-document parsing into a unified graph representation lacks well-defined representation and evaluation. Taking advantage of a super-sentential level of coreference annotation from previous work, we introduce a simple algorithm for deriving a unified graph representation, avoiding the pitfalls of information loss from over-merging and lack of coherence from under-merging. Next, we describe improvements to the Smatch metric to make it tractable for comparing document-level graphs, and use it to re-evaluate the best published document-level AMR parser. We also present a pipeline approach combining the top performing AMR parser and coreference resolution systems, providing a strong baseline for future research.