Probabilistic Generative Transformer Language models for Generative Design of Molecules
This work addresses the need for interpretable and data-efficient generative models for molecule design, which is incremental as it adapts an existing text-based language model to the molecular domain.
The authors tackled the problem of generative molecule design by proposing GMTransformer, a probabilistic neural network model that achieves high novelty and scaffold similarity (Scaf) on the MOSES datasets compared to other baselines.
Self-supervised neural language models have recently found wide applications in generative design of organic molecules and protein sequences as well as representation learning for downstream structure classification and functional prediction. However, most of the existing deep learning models for molecule design usually require a big dataset and have a black-box architecture, which makes it difficult to interpret their design logic. Here we propose Generative Molecular Transformer (GMTransformer), a probabilistic neural network model for generative design of molecules. Our model is built on the blank filling language model originally developed for text processing, which has demonstrated unique advantages in learning the "molecules grammars" with high-quality generation, interpretability, and data efficiency. Benchmarked on the MOSES datasets, our models achieve high novelty and Scaf compared to other baselines. The probabilistic generation steps have the potential in tinkering molecule design due to their capability of recommending how to modify existing molecules with explanation, guided by the learned implicit molecule chemistry. The source code and datasets can be accessed freely at https://github.com/usccolumbia/GMTransformer