MolMetaLM: a Physicochemical Knowledge-Guided Molecular Meta Language Model
This work addresses the need for more accurate and semantically rich molecular representations in drug discovery and materials science, offering a novel approach that integrates domain-specific knowledge.
The authors tackled the problem of molecular language models lacking physicochemical knowledge by proposing MolMetaLM, a meta language framework that uses knowledge triples to pretrain on diverse tasks, achieving proficiency in property prediction, molecule generation, conformation inference, and molecular optimization across large-scale benchmarks.
Most current molecular language models transfer the masked language model or image-text generation model from natural language processing to molecular field. However, molecules are not solely characterized by atom/bond symbols; they encapsulate important physical/chemical properties. Moreover, normal language models bring grammar rules that are irrelevant for understanding molecules. In this study, we propose a novel physicochemical knowledge-guided molecular meta language framework MolMetaLM. We design a molecule-specialized meta language paradigm, formatted as multiple <S,P,O> (subject, predicate, object) knowledge triples sharing the same S (i.e., molecule) to enhance learning the semantic relationships between physicochemical knowledge and molecules. By introducing different molecular knowledge and noises, the meta language paradigm generates tens of thousands of pretraining tasks. By recovering the token/sequence/order-level noises, MolMetaLM exhibits proficiency in large-scale benchmark evaluations involving property prediction, molecule generation, conformation inference, and molecular optimization. Through MolMetaLM, we offer a new insight for designing language models.