AMR Quality Rating with a Lightweight CNN
This work addresses the need for reliable AMR quality assessment in NLP tasks, though it is incremental as it adapts existing CNN methods to a new domain.
The paper tackles the problem of varying quality in automatic Abstract Meaning Representation (AMR) parses by proposing a lightweight CNN that rates AMR quality without gold data, achieving more accurate ratings than strong baselines and reducing energy consumption.
Structured semantic sentence representations such as Abstract Meaning Representations (AMRs) are potentially useful in various NLP tasks. However, the quality of automatic parses can vary greatly and jeopardizes their usefulness. This can be mitigated by models that can accurately rate AMR quality in the absence of costly gold data, allowing us to inform downstream systems about an incorporated parse's trustworthiness or select among different candidate parses. In this work, we propose to transfer the AMR graph to the domain of images. This allows us to create a simple convolutional neural network (CNN) that imitates a human judge tasked with rating graph quality. Our experiments show that the method can rate quality more accurately than strong baselines, in several quality dimensions. Moreover, the method proves to be efficient and reduces the incurred energy consumption.