SkipNode: On Alleviating Performance Degradation for Deep Graph Convolutional Networks
This addresses a critical bottleneck for researchers and practitioners using deep GCNs in graph-based tasks, offering a novel solution to improve model depth without performance loss.
The paper tackles the performance degradation problem in deep Graph Convolutional Networks (GCNs) by identifying over-smoothing and gradient vanishing as mutually reinforcing causes, and proposes SkipNode, a plug-and-play module that samples nodes to skip convolutions, achieving superior results over state-of-the-art baselines.
Graph Convolutional Networks (GCNs) suffer from performance degradation when models go deeper. However, earlier works only attributed the performance degeneration to over-smoothing. In this paper, we conduct theoretical and experimental analysis to explore the fundamental causes of performance degradation in deep GCNs: over-smoothing and gradient vanishing have a mutually reinforcing effect that causes the performance to deteriorate more quickly in deep GCNs. On the other hand, existing anti-over-smoothing methods all perform full convolutions up to the model depth. They could not well resist the exponential convergence of over-smoothing due to model depth increasing. In this work, we propose a simple yet effective plug-and-play module, Skipnode, to overcome the performance degradation of deep GCNs. It samples graph nodes in each convolutional layer to skip the convolution operation. In this way, both over-smoothing and gradient vanishing can be effectively suppressed since (1) not all nodes'features propagate through full layers and, (2) the gradient can be directly passed back through ``skipped'' nodes. We provide both theoretical analysis and empirical evaluation to demonstrate the efficacy of Skipnode and its superiority over SOTA baselines.