Improved Semantic Role Labeling using Parameterized Neighborhood Memory Adaptation
This work offers an incremental improvement in semantic role labeling performance for researchers and practitioners working with natural language understanding, especially for out-of-domain text.
This paper addresses the challenge of improving semantic role labeling (SRL) performance in deep neural models, particularly for low-frequency and out-of-domain contexts. The proposed Parameterized Neighborhood Memory Adaptive (PNMA) method, when combined with BERT embeddings, achieved F1 scores of 80.2 on CoNLL2005 and 84.97 on CoNLL2009 datasets, outperforming existing models.
Deep neural models achieve some of the best results for semantic role labeling. Inspired by instance-based learning that utilizes nearest neighbors to handle low-frequency context-specific training samples, we investigate the use of memory adaptation techniques in deep neural models. We propose a parameterized neighborhood memory adaptive (PNMA) method that uses a parameterized representation of the nearest neighbors of tokens in a memory of activations and makes predictions based on the most similar samples in the training data. We empirically show that PNMA consistently improves the SRL performance of the base model irrespective of types of word embeddings. Coupled with contextualized word embeddings derived from BERT, PNMA improves over existing models for both span and dependency semantic parsing datasets, especially on out-of-domain text, reaching F1 scores of 80.2, and 84.97 on CoNLL2005, and CoNLL2009 datasets, respectively.