Motif-aware Attribute Masking for Molecular Graph Pre-training
This work addresses a bottleneck in molecular graph pre-training for chemistry, biomedicine, and material science applications, but it is incremental as it builds on existing attribute masking strategies.
The paper tackled the problem of graph neural networks over-relying on local neighbors during attribute masking pre-training, which inhibits learning from higher-level substructures like chemical motifs, and proposed motif-aware attribute masking to capture inter-motif structures, showing advantages on eight molecular property prediction datasets.
Attribute reconstruction is used to predict node or edge features in the pre-training of graph neural networks. Given a large number of molecules, they learn to capture structural knowledge, which is transferable for various downstream property prediction tasks and vital in chemistry, biomedicine, and material science. Previous strategies that randomly select nodes to do attribute masking leverage the information of local neighbors However, the over-reliance of these neighbors inhibits the model's ability to learn from higher-level substructures. For example, the model would learn little from predicting three carbon atoms in a benzene ring based on the other three but could learn more from the inter-connections between the functional groups, or called chemical motifs. In this work, we propose and investigate motif-aware attribute masking strategies to capture inter-motif structures by leveraging the information of atoms in neighboring motifs. Once each graph is decomposed into disjoint motifs, the features for every node within a sample motif are masked. The graph decoder then predicts the masked features of each node within the motif for reconstruction. We evaluate our approach on eight molecular property prediction datasets and demonstrate its advantages.