On the Trade-off between Over-smoothing and Over-squashing in Deep Graph Neural Networks
This addresses a key bottleneck for researchers and practitioners using deep GNNs in computer science applications, though it is incremental as it builds on existing curvature-based methods.
The paper tackles the problem of over-smoothing and over-squashing in deep Graph Neural Networks (GNNs), which hinder performance, by revealing their intrinsic relation to the spectral gap and proposing the SJLR algorithm for edge rewiring, achieving competitive results.
Graph Neural Networks (GNNs) have succeeded in various computer science applications, yet deep GNNs underperform their shallow counterparts despite deep learning's success in other domains. Over-smoothing and over-squashing are key challenges when stacking graph convolutional layers, hindering deep representation learning and information propagation from distant nodes. Our work reveals that over-smoothing and over-squashing are intrinsically related to the spectral gap of the graph Laplacian, resulting in an inevitable trade-off between these two issues, as they cannot be alleviated simultaneously. To achieve a suitable compromise, we propose adding and removing edges as a viable approach. We introduce the Stochastic Jost and Liu Curvature Rewiring (SJLR) algorithm, which is computationally efficient and preserves fundamental properties compared to previous curvature-based methods. Unlike existing approaches, SJLR performs edge addition and removal during GNN training while maintaining the graph unchanged during testing. Comprehensive comparisons demonstrate SJLR's competitive performance in addressing over-smoothing and over-squashing.