Semantic-aware Contrastive Learning for More Accurate Semantic Parsing
This addresses the problem of improving semantic parsing accuracy for natural language processing applications, representing an incremental advancement through a novel method for a known bottleneck.
The paper tackles the difficulty of training discriminative semantic parsers via Maximum Likelihood Estimation by proposing a semantic-aware contrastive learning algorithm that learns to distinguish fine-grained meaning representations, achieving state-of-the-art performances on two standard datasets.
Since the meaning representations are detailed and accurate annotations which express fine-grained sequence-level semtantics, it is usually hard to train discriminative semantic parsers via Maximum Likelihood Estimation (MLE) in an autoregressive fashion. In this paper, we propose a semantic-aware contrastive learning algorithm, which can learn to distinguish fine-grained meaning representations and take the overall sequence-level semantic into consideration. Specifically, a multi-level online sampling algorithm is proposed to sample confusing and diverse instances. Three semantic-aware similarity functions are designed to accurately measure the distance between meaning representations as a whole. And a ranked contrastive loss is proposed to pull the representations of the semantic-identical instances together and push negative instances away. Experiments on two standard datasets show that our approach achieves significant improvements over MLE baselines and gets state-of-the-art performances by simply applying semantic-aware contrastive learning on a vanilla Seq2Seq model.