RLDBF: Enhancing LLMs Via Reinforcement Learning With DataBase FeedBack
This addresses the challenge of integrating structured scientific databases into LLMs for AI in Science, representing a novel method rather than an incremental improvement.
This paper tackles the problem of large language models (LLMs) being inadequate at leveraging structured scientific data like chemical molecular properties, proposing Reinforcement Learning with Database Feedback (RLDBF) to enhance LLMs. The model demonstrated remarkable generalization capabilities on unseen data and other chemical tasks, substantiating its potential for structured scientific data processing.
While current large language models (LLMs) demonstrate remarkable linguistic capabilities through training on massive unstructured text corpora, they remain inadequate in leveraging structured scientific data (e.g., chemical molecular properties in databases) that encapsulate centuries of accumulated scientific expertise. These structured datasets hold strategic significance for advancing AI for Science yet current approaches merely treat them as auxiliary supplements to unstructured text. This study pioneers a systematic investigation into enhancing LLMs with structured scientific data, using chemical molecular science as a testbed. We investigate the impact of incorporating molecular property data on LLM across distinct training phases, including continual pre-training, supervised fine-tuning, and reinforcement learning. Notably, to address the inherent limitation of numerical insensitivity in large models, we propose an innovative methodology termed "Reinforcement Learning with Database Feedback" (RLDBF). Experimental evaluations demonstrate the efficacy of the proposed approach, with the model exhibiting remarkable generalization capabilities on previously unseen data and other chemical tasks. The results substantiate the potential of our method in advancing the field of structured scientific data processing within LLMs.