LLamol: A Dynamic Multi-Conditional Generative Transformer for De Novo Molecular Design
This work provides a potent and expandable tool for researchers in chemistry and drug discovery to design new molecules with specific properties, though it is incremental as it builds on existing transformer architectures and training data.
The authors tackled the problem of de novo molecular design by developing LLamol, a dynamic multi-conditional generative transformer based on LLama 2, trained on 13M organic compounds, which generates valid molecular structures in SMILES notation while flexibly incorporating up to four conditions, such as numerical properties and token sequences, with satisfactory results in all tested scenarios.
Generative models have demonstrated substantial promise in Natural Language Processing (NLP) and have found application in designing molecules, as seen in General Pretrained Transformer (GPT) models. In our efforts to develop such a tool for exploring the organic chemical space in search of potentially electro-active compounds, we present "LLamol", a single novel generative transformer model based on the LLama 2 architecture, which was trained on a 13M superset of organic compounds drawn from diverse public sources. To allow for a maximum flexibility in usage and robustness in view of potentially incomplete data, we introduce "Stochastic Context Learning" as a new training procedure. We demonstrate that the resulting model adeptly handles single- and multi-conditional organic molecule generation with up to four conditions, yet more are possible. The model generates valid molecular structures in SMILES notation while flexibly incorporating three numerical and/or one token sequence into the generative process, just as requested. The generated compounds are very satisfactory in all scenarios tested. In detail, we showcase the model's capability to utilize token sequences for conditioning, either individually or in combination with numerical properties, making LLamol a potent tool for de novo molecule design, easily expandable with new properties.