Automated Text Mining of Experimental Methodologies from Biomedical Literature
This work addresses the need for efficient text mining in biomedical literature, though it appears incremental as it builds on existing pre-trained models.
The authors tackled the problem of classifying biomedical texts by proposing a fine-tuned DistilBERT model, which reduced model size by 40% and was 60% faster while outperforming traditional methods like RNN or LSTM.
Biomedical literature is a rapidly expanding field of science and technology. Classification of biomedical texts is an essential part of biomedicine research, especially in the field of biology. This work proposes the fine-tuned DistilBERT, a methodology-specific, pre-trained generative classification language model for mining biomedicine texts. The model has proven its effectiveness in linguistic understanding capabilities and has reduced the size of BERT models by 40\% but by 60\% faster. The main objective of this project is to improve the model and assess the performance of the model compared to the non-fine-tuned model. We used DistilBert as a support model and pre-trained on a corpus of 32,000 abstracts and complete text articles; our results were impressive and surpassed those of traditional literature classification methods by using RNN or LSTM. Our aim is to integrate this highly specialised and specific model into different research industries.