Impact of translation on biomedical information extraction from real-life clinical notes
This addresses the challenge of information extraction in non-English medical texts for healthcare and NLP researchers, showing that translation-based approaches are less effective, which is an incremental finding.
The study tackled the problem of extracting biomedical information from French clinical notes by comparing native French models to English models applied to translations, finding that the native French method performed better with an F1 score of 0.51 compared to 0.39 and 0.38 for the English methods.
The objective of our study is to determine whether using English tools to extract and normalize French medical concepts on translations provides comparable performance to French models trained on a set of annotated French clinical notes. We compare two methods: a method involving French language models and a method involving English language models. For the native French method, the Named Entity Recognition (NER) and normalization steps are performed separately. For the translated English method, after the first translation step, we compare a two-step method and a terminology-oriented method that performs extraction and normalization at the same time. We used French, English and bilingual annotated datasets to evaluate all steps (NER, normalization and translation) of our algorithms. Concerning the results, the native French method performs better than the translated English one with a global f1 score of 0.51 [0.47;0.55] against 0.39 [0.34;0.44] and 0.38 [0.36;0.40] for the two English methods tested. In conclusion, despite the recent improvement of the translation models, there is a significant performance difference between the two approaches in favor of the native French method which is more efficient on French medical texts, even with few annotated documents.