BioAlchemy: Distilling Biological Literature into Reasoning-Ready Reinforcement Learning Training Data
For AI researchers and biologists, this work provides a method to generate domain-aligned training data for improving reasoning in biology, though it is an incremental application of existing techniques.
BioAlchemy addresses the misalignment between existing reasoning datasets and modern biology research topics by creating a pipeline to extract verifiable QA pairs from scientific literature. Using the resulting 345K dataset with reinforcement learning, they achieve a 9.12% improvement on biology benchmarks with their 8B model.
Despite the large corpus of biology training text, the impact of reasoning models on biological research generally lags behind math and coding. In this work, we show that biology questions from current large-scale reasoning datasets do not align well with modern research topic distributions in biology, and that this topic imbalance may negatively affect performance. In addition, we find that methods for extracting challenging and verifiable research problems from biology research text are a critical yet underdeveloped ingredient in applying reinforcement learning for better performance on biology research tasks. We introduce BioAlchemy, a pipeline for sourcing a diverse set of verifiable question-and-answer pairs from a scientific corpus of biology research text. We curate BioAlchemy-345K, a training dataset containing over 345K scientific reasoning problems in biology. Then, we demonstrate how aligning our dataset to the topic distribution of modern scientific biology can be used with reinforcement learning to improve reasoning performance. Finally, we present BioAlchemist-8B, which improves over its base reasoning model by 9.12% on biology benchmarks. These results demonstrate the efficacy of our approach for developing stronger scientific reasoning capabilities in biology. The BioAlchemist-8B model is available at: https://huggingface.co/BioAlchemy.