Teaching and Evaluating LLMs to Reason About Polymer Design Related Tasks
This work addresses the problem of AI4Science applications in polymer design for researchers, offering an incremental improvement through a specialized dataset and training method.
The authors tackled the ineffectiveness of current LLMs in polymer design by creating PolyBench, a large-scale dataset of over 125K tasks, and a knowledge-augmented reasoning distillation method, resulting in small language models (7B-14B parameters) outperforming similar-sized and frontier LLMs on polymer design benchmarks.
Research in AI4Science has shown promise in many science applications, including polymer design. However, current LLMs prove ineffective on this problem space because: (i) most models lack polymer-specific knowledge (ii) existing aligned models lack coverage of knowledge and capabilities relevant to polymer design. Addressing this, we introduce PolyBench, a large scale training and test benchmark dataset of more than 125K polymer design related tasks, leveraging a knowledge base of 13M+ data points obtained from experimental and synthetic sources to ensure broad coverage of polymers and their properties. For effective alignment using PolyBench, we introduce a knowledge-augmented reasoning distillation method that augments this dataset with structured CoT. Furthermore, tasks in PolyBench are organized from simple to complex analytical reasoning problems, enabling generalization tests and diagnostic probes across the problem space. Experiments show that small language models (SLMs), of 7B to 14B parameters, trained on PolyBench data outperform similar sized models, and even closed source frontier LLMs on PolyBench test dataset while demonstrating gains on other polymer benchmarks as well.