SMSMix: Sense-Maintained Sentence Mixup for Word Sense Disambiguation
This addresses a specific issue in NLP for WSD systems, but it is incremental as it builds on existing mixup techniques by adapting them to preserve word sense.
The paper tackles the problem of poor performance on rare word senses in Word Sense Disambiguation due to nonuniform sense distributions during training, and proposes SMSMix, a data augmentation method that increases the frequency of least frequent senses, resulting in improved performance as validated through experiments.
Word Sense Disambiguation (WSD) is an NLP task aimed at determining the correct sense of a word in a sentence from discrete sense choices. Although current systems have attained unprecedented performances for such tasks, the nonuniform distribution of word senses during training generally results in systems performing poorly on rare senses. To this end, we consider data augmentation to increase the frequency of these least frequent senses (LFS) to reduce the distributional bias of senses during training. We propose Sense-Maintained Sentence Mixup (SMSMix), a novel word-level mixup method that maintains the sense of a target word. SMSMix smoothly blends two sentences using mask prediction while preserving the relevant span determined by saliency scores to maintain a specific word's sense. To the best of our knowledge, this is the first attempt to apply mixup in NLP while preserving the meaning of a specific word. With extensive experiments, we validate that our augmentation method can effectively give more information about rare senses during training with maintained target sense label.