Is neural semantic parsing good at ellipsis resolution, or isn't it?
This addresses a specific challenge in natural language processing for improving semantic parsing accuracy in linguistically complex contexts, though it is incremental as it focuses on a particular phenomenon.
The paper tackled the problem of neural semantic parsers' performance on English verb phrase ellipsis, a context-sensitive linguistic phenomenon, and found that while parsers achieved over 90% semantic matching on standard tests, they failed on a constructed corpus of 120 ellipsis cases, with data augmentation improving results.
Neural semantic parsers have shown good overall performance for a variety of linguistic phenomena, reaching semantic matching scores of more than 90%. But how do such parsers perform on strongly context-sensitive phenomena, where large pieces of semantic information need to be duplicated to form a meaningful semantic representation? A case in point is English verb phrase ellipsis, a construct where entire verb phrases can be abbreviated by a single auxiliary verb. Are the otherwise known as powerful semantic parsers able to deal with ellipsis or aren't they? We constructed a corpus of 120 cases of ellipsis with their fully resolved meaning representation and used this as a challenge set for a large battery of neural semantic parsers. Although these parsers performed very well on the standard test set, they failed in the instances with ellipsis. Data augmentation helped improve the parsing results. The reason for the difficulty of parsing elided phrases is not that copying semantic material is hard, but that usually occur in linguistically complicated contexts causing most of the parsing errors.