BERT-based Acronym Disambiguation with Multiple Training Strategies
This addresses a practical NLP problem for scientific document processing, but it is incremental as it builds on existing BERT methods with added strategies.
The paper tackles the problem of acronym disambiguation in scientific texts by proposing a BERT-based binary classification model with multiple training strategies, achieving first place in the SDU@AAAI-21 shared task.
Acronym disambiguation (AD) task aims to find the correct expansions of an ambiguous ancronym in a given sentence. Although it is convenient to use acronyms, sometimes they could be difficult to understand. Identifying the appropriate expansions of an acronym is a practical task in natural language processing. Since few works have been done for AD in scientific field, we propose a binary classification model incorporating BERT and several training strategies including dynamic negative sample selection, task adaptive pretraining, adversarial training and pseudo labeling in this paper. Experiments on SciAD show the effectiveness of our proposed model and our score ranks 1st in SDU@AAAI-21 shared task 2: Acronym Disambiguation.