How Wide and How Deep? Mitigating Over-Squashing of GNNs via Channel Capacity Constrained Estimation
This addresses a specific bottleneck in graph neural networks for researchers and practitioners, offering a method to mitigate information loss during propagation, though it appears incremental as it builds on existing spectral GNN models.
The paper tackled the problem of over-squashing in graph neural networks by proposing Channel Capacity Constrained Estimation (C3E), a framework that formulates hidden dimension and depth selection as a nonlinear programming problem based on information theory, resulting in improved representation learning across nine public datasets.
Existing graph neural networks typically rely on heuristic choices for hidden dimensions and propagation depths, which often lead to severe information loss during propagation, known as over-squashing. To address this issue, we propose Channel Capacity Constrained Estimation (C3E), a novel framework that formulates the selection of hidden dimensions and depth as a nonlinear programming problem grounded in information theory. Through modeling spectral graph neural networks as communication channels, our approach directly connects channel capacity to hidden dimensions, propagation depth, propagation mechanism, and graph structure. Extensive experiments on nine public datasets demonstrate that hidden dimensions and depths estimated by C3E can mitigate over-squashing and consistently improve representation learning. Experimental results show that over-squashing occurs due to the cumulative compression of information in representation matrices. Furthermore, our findings show that increasing hidden dimensions indeed mitigate information compression, while the role of propagation depth is more nuanced, uncovering a fundamental balance between information compression and representation complexity.