Mol-LLM: Multimodal Generalist Molecular LLM with Improved Graph Utilization
This work addresses the need for better multimodal generalist models in molecular AI, offering significant gains for tasks like chemical reaction and property prediction, though it is incremental in advancing existing multimodal approaches.
The paper tackles the problem of limited graph-structural information utilization in multimodal molecular LLMs by proposing Molecular structure Preference Optimization (MolPO) and an advanced graph encoder, resulting in Mol-LLM achieving state-of-the-art or comparable results on comprehensive benchmarks, including large improvements on out-of-distribution datasets.
Recent advances in large language models (LLMs) have led to models that tackle diverse molecular tasks, such as chemical reaction prediction and molecular property prediction. Large-scale molecular instruction-tuning datasets have enabled sequence-only (e.g., SMILES or SELFIES) generalist molecular LLMs, and researchers are now exploring multimodal approaches that incorporate molecular structural information for further gains. However, a genuinely multimodal, generalist LLM that covers a broad spectrum of molecular tasks has yet to be fully investigated. We observe that naive next token prediction training ignores graph-structural information, limiting an LLM's ability to exploit molecular graphs. To address this, we propose (i) Molecular structure Preference Optimization (MolPO), which facilitates graph usage by optimizing preferences between pairs of correct and perturbed molecular structures, and (ii) an advanced graph encoder with a tailored pre-training strategy to improve the effect of graph utilization by MolPO. Building on these contributions, we introduce Mol-LLM, the first multimodal generalist model that (a) handles a broad spectrum of molecular tasks among molecular LLMs, (b) explicitly leverages molecular-structure information, and (c) takes advantage of extensive instruction tuning. Mol-LLM attains state-of-the-art or comparable results across the most comprehensive molecular-LLM benchmark-even on out-of-distribution datasets for reaction and property prediction, where it surpasses prior generalist molecular LLMs by a large margin.