Training Bi-Encoders for Word Sense Disambiguation
This work addresses the problem of word sense ambiguity in NLP, offering incremental improvements for tasks requiring precise semantic understanding.
The paper tackles Word Sense Disambiguation by optimizing bi-encoders with alternative lexical information presentation and a multi-stage pipeline, achieving state-of-the-art results on standard benchmarks.
Modern transformer-based neural architectures yield impressive results in nearly every NLP task and Word Sense Disambiguation, the problem of discerning the correct sense of a word in a given context, is no exception. State-of-the-art approaches in WSD today leverage lexical information along with pre-trained embeddings from these models to achieve results comparable to human inter-annotator agreement on standard evaluation benchmarks. In the same vein, we experiment with several strategies to optimize bi-encoders for this specific task and propose alternative methods of presenting lexical information to our model. Through our multi-stage pre-training and fine-tuning pipeline we further the state of the art in Word Sense Disambiguation.