Scientific production in the era of Large Language Models
This research addresses how LLMs are transforming scientific production, with implications for journals, funding agencies, and tenure committees, though it is incremental in documenting existing trends.
The study analyzed the impact of Large Language Models (LLMs) on scientific research using datasets including 2.1M preprints, finding that LLM adoption increases paper production by 23.7-89.3% but leads to linguistically complex yet substantively underwhelming manuscripts, while also diversifying citations.
Large Language Models (LLMs) are rapidly reshaping scientific research. We analyze these changes in multiple, large-scale datasets with 2.1M preprints, 28K peer review reports, and 246M online accesses to scientific documents. We find: 1) scientists adopting LLMs to draft manuscripts demonstrate a large increase in paper production, ranging from 23.7-89.3% depending on scientific field and author background, 2) LLM use has reversed the relationship between writing complexity and paper quality, leading to an influx of manuscripts that are linguistically complex but substantively underwhelming, and 3) LLM adopters access and cite more diverse prior work, including books and younger, less-cited documents. These findings highlight a stunning shift in scientific production that will likely require a change in how journals, funding agencies, and tenure committees evaluate scientific works.