A Survey on Oversmoothing in Graph Neural Networks
This is a survey that synthesizes and extends existing knowledge on a known bottleneck in GNNs, making it incremental but useful for researchers in graph learning.
The paper tackles the problem of over-smoothing in graph neural networks (GNNs), where node features become too similar with increased depth, by axiomatically defining it and proposing new quantitative measures, while empirically testing mitigation approaches on real-world datasets.
Node features of graph neural networks (GNNs) tend to become more similar with the increase of the network depth. This effect is known as over-smoothing, which we axiomatically define as the exponential convergence of suitable similarity measures on the node features. Our definition unifies previous approaches and gives rise to new quantitative measures of over-smoothing. Moreover, we empirically demonstrate this behavior for several over-smoothing measures on different graphs (small-, medium-, and large-scale). We also review several approaches for mitigating over-smoothing and empirically test their effectiveness on real-world graph datasets. Through illustrative examples, we demonstrate that mitigating over-smoothing is a necessary but not sufficient condition for building deep GNNs that are expressive on a wide range of graph learning tasks. Finally, we extend our definition of over-smoothing to the rapidly emerging field of continuous-time GNNs.