Curriculum Learning for Graph Neural Networks: Which Edges Should We Learn First
This work addresses a domain-specific problem for graph learning practitioners by introducing a novel method to enhance GNN training, though it is incremental as it adapts existing curriculum learning principles to graph data.
The paper tackles the problem of Graph Neural Networks (GNNs) treating all edges equally, which can lead to suboptimal representations due to varying edge difficulty and noise, by proposing a curriculum learning strategy that gradually incorporates edges from easy to hard based on training status, resulting in improved generalization and robustness across nine synthetic and nine real-world datasets.
Graph Neural Networks (GNNs) have achieved great success in representing data with dependencies by recursively propagating and aggregating messages along the edges. However, edges in real-world graphs often have varying degrees of difficulty, and some edges may even be noisy to the downstream tasks. Therefore, existing GNNs may lead to suboptimal learned representations because they usually treat every edge in the graph equally. On the other hand, Curriculum Learning (CL), which mimics the human learning principle of learning data samples in a meaningful order, has been shown to be effective in improving the generalization ability and robustness of representation learners by gradually proceeding from easy to more difficult samples during training. Unfortunately, existing CL strategies are designed for independent data samples and cannot trivially generalize to handle data dependencies. To address these issues, we propose a novel CL strategy to gradually incorporate more edges into training according to their difficulty from easy to hard, where the degree of difficulty is measured by how well the edges are expected given the model training status. We demonstrate the strength of our proposed method in improving the generalization ability and robustness of learned representations through extensive experiments on nine synthetic datasets and nine real-world datasets. The code for our proposed method is available at https://github.com/rollingstonezz/Curriculum_learning_for_GNNs.