Empirical Evidence for the Fragment level Understanding on Drug Molecular Structure of LLMs
This addresses the problem of understanding AI's interpretability in drug discovery for researchers, but it is incremental as it builds on existing SMILES-based language models.
The study investigated whether language models can understand chemical spatial structures from 1D SMILES sequences by pre-training a transformer on chemical language and fine-tuning it for drug design, finding that the models learn structural knowledge reflected in high-frequency SMILES substrings.
AI for drug discovery has been a research hotspot in recent years, and SMILES-based language models has been increasingly applied in drug molecular design. However, no work has explored whether and how language models understand the chemical spatial structure from 1D sequences. In this work, we pre-train a transformer model on chemical language and fine-tune it toward drug design objectives, and investigate the correspondence between high-frequency SMILES substrings and molecular fragments. The results indicate that language models can understand chemical structures from the perspective of molecular fragments, and the structural knowledge learned through fine-tuning is reflected in the high-frequency SMILES substrings generated by the model.