Comment on "A Note on Over-Smoothing for Graph Neural Networks"
This work addresses over-smoothing in GNNs, a key issue for practitioners in graph-based machine learning, but it is incremental as it builds on prior analysis.
The authors analyze over-smoothing in Graph Neural Networks by showing that under mild spectral conditions, the Dirichlet energy of node embeddings decreases exponentially with depth, and they extend this to spectral polynomial filters with experiments illustrating conditions where energy increases.
We comment on Cai and Wang (2020, arXiv:2006.13318), who analyze over-smoothing in GNNs via Dirichlet energy. We show that under mild spectral conditions (including with Leaky-ReLU), the Dirichlet energy of node embeddings decreases exponentially with depth; we further extend the result to spectral polynomial filters and provide a short proof for the Leaky-ReLU case. Experiments on edge deletion and weight amplification illustrate when Dirichlet energy increases, hinting at practical ways to relieve over-smoothing.