HyPE-GT: where Graph Transformers meet Hyperbolic Positional Encodings
This work addresses the problem of improving graph representation learning for researchers and practitioners, though it is incremental as it builds on existing Graph Transformer and hyperbolic space methods.
The paper tackles the limitation of Graph Transformers ignoring node position information by introducing hyperbolic positional encodings, resulting in enhanced performance on molecular, co-author, and co-purchase network datasets.
Graph Transformers (GTs) facilitate the comprehension of graph-structured data by calculating the self-attention of node pairs without considering node position information. To address this limitation, we introduce an innovative and efficient framework that introduces Positional Encodings (PEs) into the Transformer, generating a set of learnable positional encodings in the hyperbolic space, a non-Euclidean domain. This approach empowers us to explore diverse options for optimal selection of PEs for specific downstream tasks, leveraging hyperbolic neural networks or hyperbolic graph convolutional networks. Additionally, we repurpose these positional encodings to mitigate the impact of over-smoothing in deep Graph Neural Networks (GNNs). Comprehensive experiments on molecular benchmark datasets, co-author, and co-purchase networks substantiate the effectiveness of hyperbolic positional encodings in enhancing the performance of deep GNNs.