BLCU-ICALL at SemEval-2022 Task 1: Cross-Attention Multitasking Framework for Definition Modeling
This work addresses the problem of improving definition modeling for computational linguistics researchers, but it is incremental as it builds on existing transformer and multitasking methods for a specific competition task.
The paper tackled the Definition Modeling subtrack of SemEval-2022 Task 1 by proposing a transformer-based multitasking framework with cross-attention and a masking language model objective, achieving 1st place on Italian, 2nd on Spanish and Russian, and 3rd on English and French.
This paper describes the BLCU-ICALL system used in the SemEval-2022 Task 1 Comparing Dictionaries and Word Embeddings, the Definition Modeling subtrack, achieving 1st on Italian, 2nd on Spanish and Russian, and 3rd on English and French. We propose a transformer-based multitasking framework to explore the task. The framework integrates multiple embedding architectures through the cross-attention mechanism, and captures the structure of glosses through a masking language model objective. Additionally, we also investigate a simple but effective model ensembling strategy to further improve the robustness. The evaluation results show the effectiveness of our solution. We release our code at: https://github.com/blcuicall/SemEval2022-Task1-DM.