Vocabulary Transfer for Biomedical Texts: Add Tokens if You Can Not Add Data
This addresses the challenge of achieving high accuracy in medical NLP tasks where data is limited due to privacy and accessibility issues, but it is incremental as it builds on existing vocabulary extension techniques.
The study tackled the problem of data scarcity in biomedical NLP by using vocabulary transfer to incorporate domain-specific terms, resulting in measurable improvements in model performance and inference time.
Working within specific NLP subdomains presents significant challenges, primarily due to a persistent deficit of data. Stringent privacy concerns and limited data accessibility often drive this shortage. Additionally, the medical domain demands high accuracy, where even marginal improvements in model performance can have profound impacts. In this study, we investigate the potential of vocabulary transfer to enhance model performance in biomedical NLP tasks. Specifically, we focus on vocabulary extension, a technique that involves expanding the target vocabulary to incorporate domain-specific biomedical terms. Our findings demonstrate that vocabulary extension, leads to measurable improvements in both downstream model performance and inference time.