A Graph Laplacian Eigenvector-based Pre-training Method for Graph Neural Networks
This work addresses a domain-specific problem for researchers and practitioners in graph machine learning, particularly in molecular property prediction, by offering an incremental improvement in pre-training methods for GNNs.
The paper tackles the challenge of pre-training graph neural networks (GNNs) to capture global and regional graph structure without oversmoothing, by proposing a Laplacian Eigenvector Learning Module (LELM) that predicts low-frequency eigenvectors of the graph Laplacian, resulting in improved performance on downstream molecular property prediction tasks.
The development of self-supervised graph pre-training methods is a crucial ingredient in recent efforts to design robust graph foundation models (GFMs). Structure-based pre-training methods are under-explored yet crucial for downstream applications which rely on underlying graph structure. In addition, pre-training traditional message passing GNNs to capture global and regional structure is often challenging due to the risk of oversmoothing as network depth increases. We address these gaps by proposing the Laplacian Eigenvector Learning Module (LELM), a novel pre-training module for graph neural networks (GNNs) based on predicting the low-frequency eigenvectors of the graph Laplacian. Moreover, LELM introduces a novel architecture that overcomes oversmoothing, allowing the GNN model to learn long-range interdependencies. Empirically, we show that models pre-trained via our framework outperform baseline models on downstream molecular property prediction tasks.