From Articles to Code: On-Demand Generation of Core Algorithms from Scientific Publications
This could reduce maintenance overhead for software developers by enabling flexible, on-demand code generation instead of static, human-maintained packages.
The paper tackled the high cost of maintaining software packages by using large language models (LLMs) to generate code from scientific publications, showing that current LLMs can reliably reproduce package functionality with performance indistinguishable from conventional libraries.
Maintaining software packages imposes significant costs due to dependency management, bug fixes, and versioning. We show that rich method descriptions in scientific publications can serve as standalone specifications for modern large language models (LLMs), enabling on-demand code generation that could supplant human-maintained libraries. We benchmark state-of-the-art models (GPT-o4-mini-high, Gemini Pro 2.5, Claude Sonnet 4) by tasking them with implementing a diverse set of core algorithms drawn from original publications. Our results demonstrate that current LLMs can reliably reproduce package functionality with performance indistinguishable from conventional libraries. These findings foreshadow a paradigm shift toward flexible, on-demand code generation and away from static, human-maintained packages, which will result in reduced maintenance overhead by leveraging published articles as sufficient context for the automated implementation of analytical workflows.