Transfer Learning for Neural Semantic Parsing
This work addresses data scarcity in semantic parsing for NLP applications, but it is incremental as it builds on existing multi-task and transfer learning methods.
The paper tackles the problem of insufficient training data for neural semantic parsing by proposing a multi-task sequence-to-sequence setup for transfer learning, achieving absolute accuracy gains of 1.0% to 4.4% on an in-house dataset and 2.5% to 7.0% on ATIS tasks.
The goal of semantic parsing is to map natural language to a machine interpretable meaning representation language (MRL). One of the constraints that limits full exploration of deep learning technologies for semantic parsing is the lack of sufficient annotation training data. In this paper, we propose using sequence-to-sequence in a multi-task setup for semantic parsing with a focus on transfer learning. We explore three multi-task architectures for sequence-to-sequence modeling and compare their performance with an independently trained model. Our experiments show that the multi-task setup aids transfer learning from an auxiliary task with large labeled data to a target task with smaller labeled data. We see absolute accuracy gains ranging from 1.0% to 4.4% in our in- house data set, and we also see good gains ranging from 2.5% to 7.0% on the ATIS semantic parsing tasks with syntactic and semantic auxiliary tasks.