Entropy Aware Message Passing in Graph Neural Networks
This addresses a key limitation in GNNs for graph learning tasks, though it is incremental as it builds on existing architectures.
The paper tackled the problem of oversmoothing in deep Graph Neural Networks by introducing an entropy-aware message passing term that preserves entropy in embeddings, achieving competitive performance across various datasets.
Deep Graph Neural Networks struggle with oversmoothing. This paper introduces a novel, physics-inspired GNN model designed to mitigate this issue. Our approach integrates with existing GNN architectures, introducing an entropy-aware message passing term. This term performs gradient ascent on the entropy during node aggregation, thereby preserving a certain degree of entropy in the embeddings. We conduct a comparative analysis of our model against state-of-the-art GNNs across various common datasets.