Architecture Matters: Uncovering Implicit Mechanisms in Graph Contrastive Learning
This work addresses the problem of inefficient adaptation of visual contrastive learning methods to graph domains for researchers and practitioners, highlighting the need for architecture-aware designs in GCL.
The study systematically analyzes graph contrastive learning (GCL) and finds that, unlike visual contrastive learning, GCL often does not require positive or negative samples and is less sensitive to data augmentations, with simple methods like Gaussian noise performing well. It provides theoretical insights into these properties by examining the implicit inductive bias of graph neural networks (GNNs).
With the prosperity of contrastive learning for visual representation learning (VCL), it is also adapted to the graph domain and yields promising performance. However, through a systematic study of various graph contrastive learning (GCL) methods, we observe that some common phenomena among existing GCL methods that are quite different from the original VCL methods, including 1) positive samples are not a must for GCL; 2) negative samples are not necessary for graph classification, neither for node classification when adopting specific normalization modules; 3) data augmentations have much less influence on GCL, as simple domain-agnostic augmentations (e.g., Gaussian noise) can also attain fairly good performance. By uncovering how the implicit inductive bias of GNNs works in contrastive learning, we theoretically provide insights into the above intriguing properties of GCL. Rather than directly porting existing VCL methods to GCL, we advocate for more attention toward the unique architecture of graph learning and consider its implicit influence when designing GCL methods. Code is available at https: //github.com/PKU-ML/ArchitectureMattersGCL.