XL-DURel: Finetuning Sentence Transformers for Ordinal Word-in-Context Classification
This work addresses the challenge of Word-in-Context classification for NLP researchers, offering a unified approach that improves performance on both ordinal and binary tasks, though it is incremental in nature.
The paper tackled the problem of ordinal Word-in-Context classification by proposing XL-DURel, a finetuned multilingual Sentence Transformer model, which outperformed previous models on ordinal and binary data using a ranking objective based on angular distance in complex space.
We propose XL-DURel, a finetuned, multilingual Sentence Transformer model optimized for ordinal Word-in-Context classification. We test several loss functions for regression and ranking tasks managing to outperform previous models on ordinal and binary data with a ranking objective based on angular distance in complex space. We further show that binary WiC can be treated as a special case of ordinal WiC and that optimizing models for the general ordinal task improves performance on the more specific binary task. This paves the way for a unified treatment of WiC modeling across different task formulations.