Character-level Representations Improve DRS-based Semantic Parsing Even in the Age of BERT
This work addresses semantic parsing for natural language processing applications, but it is incremental as it builds on existing sequence-to-sequence models with character-level enhancements.
The paper tackled the problem of Discourse Representation Structure parsing by combining character-level and contextual language model representations, resulting in robust performance improvements across different languages and datasets, with larger gains in English compared to adding linguistic information or non-contextual embeddings.
We combine character-level and contextual language model representations to improve performance on Discourse Representation Structure parsing. Character representations can easily be added in a sequence-to-sequence model in either one encoder or as a fully separate encoder, with improvements that are robust to different language models, languages and data sets. For English, these improvements are larger than adding individual sources of linguistic information or adding non-contextual embeddings. A new method of analysis based on semantic tags demonstrates that the character-level representations improve performance across a subset of selected semantic phenomena.