Enhancing Idiomatic Representation in Multiple Languages via an Adaptive Contrastive Triplet Loss
This addresses the problem of idiomatic expression modeling for NLP tasks like machine translation, though it appears incremental as it builds on existing contrastive learning techniques.
The paper tackles the challenge of modeling idiomatic language in NLP by proposing an adaptive contrastive triplet loss method, which significantly outperforms previous alternatives on a SemEval challenge across many metrics.
Accurately modeling idiomatic or non-compositional language has been a longstanding challenge in Natural Language Processing (NLP). This is partly because these expressions do not derive their meanings solely from their constituent words, but also due to the scarcity of relevant data resources, and their impact on the performance of downstream tasks such as machine translation and simplification. In this paper we propose an approach to model idiomaticity effectively using a triplet loss that incorporates the asymmetric contribution of components words to an idiomatic meaning for training language models by using adaptive contrastive learning and resampling miners to build an idiomatic-aware learning objective. Our proposed method is evaluated on a SemEval challenge and outperforms previous alternatives significantly in many metrics.