Residual connections provably mitigate oversmoothing in graph neural networks
This addresses a critical bottleneck for practitioners using deep GNNs in domains like social networks or bioinformatics, though it is incremental as it builds on existing residual connection techniques.
The paper tackles the problem of oversmoothing in deep graph neural networks, where vertex features become indistinguishable, and shows that adding residual connections effectively mitigates or prevents this issue, with theoretical and numerical support.
Graph neural networks (GNNs) have achieved remarkable empirical success in processing and representing graph-structured data across various domains. However, a significant challenge known as "oversmoothing" persists, where vertex features become nearly indistinguishable in deep GNNs, severely restricting their expressive power and practical utility. In this work, we analyze the asymptotic oversmoothing rates of deep GNNs with and without residual connections by deriving explicit convergence rates for a normalized vertex similarity measure. Our analytical framework is grounded in the multiplicative ergodic theorem. Furthermore, we demonstrate that adding residual connections effectively mitigates or prevents oversmoothing across several broad families of parameter distributions. The theoretical findings are strongly supported by numerical experiments.