An Empirical Investigation of Pre-Trained Deep Learning Model Reuse in the Scientific Process
It addresses the challenge for natural scientists by providing the first empirical analysis of PTM reuse practices, though it is incremental as it builds on prior recommendations without introducing new methods.
This paper tackles the problem of high technical costs in training deep learning models for natural sciences by empirically studying the reuse of pre-trained models (PTMs), finding that 'Biochemistry, Genetics and Molecular Biology' leads in PTM reuse, 'adaptation' is the most common pattern, and the 'Test' stage is most impacted.
Deep learning has achieved recognition for its impact within natural sciences, however scientists are inhibited by the prohibitive technical cost and computational complexity of training project specific models from scratch. Following software engineering community guidance, natural scientists are reusing pre-trained deep learning models (PTMs) to amortize these costs. While prior works recommend PTM reuse patterns, to our knowledge, little work has been done to empirically evaluate their usage and impact within the natural sciences. We present the first empirical study of PTM reuse patterns in the natural sciences, quantifying the utilization and impact of conceptual, adaptation, and deployment reuse within the scientific process. Leveraging an automated large language model driven pipeline, we analyze 17,511 peer reviewed, open access papers to identify PTM reuse by scientific field, associated reuse patterns, and the impact of PTM integration into the scientific process from January 1st, 2000 to December 10th, 2025. Our results show that "Biochemistry, Genetics and Molecular Biology" has outpaced other natural scientific fields in PTM reuse, "adaptation" reuse is the most prevalent PTM reuse pattern identified across all natural science fields, and the "Test" stage of the scientific process has been most impacted by PTM integration. This aligns with the growing interest of leveraging computational methods to conduct high throughput, data driven scientific research. Our work characterizes and identifies current PTM reuse practices within the natural sciences, evaluates their impact on the scientific process, and establishes a foundation for future work into the implementation and broader scientific implications of PTM reuse.