Probing Graph Neural Network Activation Patterns Through Graph Topology
This work provides a diagnostic probe for understanding failures in graph learning, which is incremental for researchers in graph machine learning.
The study investigated how graph topology interacts with learned preferences in Graph Neural Networks, finding that extreme edge activations in Graph Transformers do not concentrate on curvature extremes, and global attention mechanisms exacerbate topological bottlenecks, increasing negative curvature prevalence on the Long Range Graph Benchmark.
Curvature notions on graphs provide a theoretical description of graph topology, highlighting bottlenecks and denser connected regions. Artifacts of the message passing paradigm in Graph Neural Networks, such as oversmoothing and oversquashing, have been attributed to these regions. However, it remains unclear how the topology of a graph interacts with the learned preferences of GNNs. Through Massive Activations, which correspond to extreme edge activation values in Graph Transformers, we probe this correspondence. Our findings on synthetic graphs and molecular benchmarks reveal that MAs do not preferentially concentrate on curvature extremes, despite their theoretical link to information flow. On the Long Range Graph Benchmark, we identify a systemic \textit{curvature shift}: global attention mechanisms exacerbate topological bottlenecks, drastically increasing the prevalence of negative curvature. Our work reframes curvature as a diagnostic probe for understanding when and why graph learning fails.