Exploring the Word Sense Disambiguation Capabilities of Large Language Models
This work addresses the problem of word sense disambiguation for computational linguistics, showing incremental improvements through fine-tuning.
The study evaluated large language models (LLMs) on word sense disambiguation tasks, finding that while LLMs perform well in zero-shot learning, they do not surpass current state-of-the-art methods, but a fine-tuned medium-sized model outperforms all others.
Word Sense Disambiguation (WSD) is a historical task in computational linguistics that has received much attention over the years. However, with the advent of Large Language Models (LLMs), interest in this task (in its classical definition) has decreased. In this study, we evaluate the performance of various LLMs on the WSD task. We extend a previous benchmark (XL-WSD) to re-design two subtasks suitable for LLM: 1) given a word in a sentence, the LLM must generate the correct definition; 2) given a word in a sentence and a set of predefined meanings, the LLM must select the correct one. The extended benchmark is built using the XL-WSD and BabelNet. The results indicate that LLMs perform well in zero-shot learning but cannot surpass current state-of-the-art methods. However, a fine-tuned model with a medium number of parameters outperforms all other models, including the state-of-the-art.