Exploring prompts to elicit memorization in masked language model-based named entity recognition
This work addresses privacy and generalization risks in language models for researchers and practitioners, though it is incremental as it builds on existing memorization analysis methods.
The paper investigates how different prompts affect the detection of training data memorization in masked language models for named entity recognition, finding that prompt performance varies by up to 16 percentage points and is model-dependent but generalizes across name sets.
Training data memorization in language models impacts model capability (generalization) and safety (privacy risk). This paper focuses on analyzing prompts' impact on detecting the memorization of 6 masked language model-based named entity recognition models. Specifically, we employ a diverse set of 400 automatically generated prompts, and a pairwise dataset where each pair consists of one person's name from the training set and another name out of the set. A prompt completed with a person's name serves as input for getting the model's confidence in predicting this name. Finally, the prompt performance of detecting model memorization is quantified by the percentage of name pairs for which the model has higher confidence for the name from the training set. We show that the performance of different prompts varies by as much as 16 percentage points on the same model, and prompt engineering further increases the gap. Moreover, our experiments demonstrate that prompt performance is model-dependent but does generalize across different name sets. A comprehensive analysis indicates how prompt performance is influenced by prompt properties, contained tokens, and the model's self-attention weights on the prompt.