Chemical Language Model Linker: blending text and molecules with modular adapters
This work addresses the challenge of efficiently generating novel molecules for drug discovery and chemical design by leveraging existing pretrained models, though it is incremental in its adapter-based approach.
The authors tackled the problem of generating molecules from text descriptions without training multi-modal models from scratch by proposing ChemLML, a lightweight adapter-based strategy that blends pretrained text and molecular models, achieving conditional molecular generation with reduced computational resources and enabling practical applications like generating protein inhibitors and membrane-permeable molecules.
The development of large language models and multi-modal models has enabled the appealing idea of generating novel molecules from text descriptions. Generative modeling would shift the paradigm from relying on large-scale chemical screening to find molecules with desired properties to directly generating those molecules. However, multi-modal models combining text and molecules are often trained from scratch, without leveraging existing high-quality pretrained models. Training from scratch consumes more computational resources and prohibits model scaling. In contrast, we propose a lightweight adapter-based strategy named Chemical Language Model Linker (ChemLML). ChemLML blends the two single domain models and obtains conditional molecular generation from text descriptions while still operating in the specialized embedding spaces of the molecular domain. ChemLML can tailor diverse pretrained text models for molecule generation by training relatively few adapter parameters. We find that the choice of molecular representation used within ChemLML, SMILES versus SELFIES, has a strong influence on conditional molecular generation performance. SMILES is often preferable despite not guaranteeing valid molecules. We raise issues in using the entire PubChem dataset of molecules and their associated descriptions for evaluating molecule generation and provide a filtered version of the dataset as a generation test set. To demonstrate how ChemLML could be used in practice, we generate candidate protein inhibitors and use docking to assess their quality and also generate candidate membrane permeable molecules.