Leveraging Manifold Embeddings for Enhanced Graph Transformer Representations and Learning
This work addresses the problem of improving graph representation learning for node classification tasks, offering an incremental enhancement to existing graph transformer methods.
The paper tackled the problem of graph transformers embedding nodes in a single Euclidean space, which blurs heterogeneous topologies, by prepending a lightweight Riemannian mixture-of-experts layer that routes nodes to appropriate manifolds (spherical, flat, hyperbolic). This approach lifted accuracy by up to 3% on four node-classification benchmarks.
Graph transformers typically embed every node in a single Euclidean space, blurring heterogeneous topologies. We prepend a lightweight Riemannian mixture-of-experts layer that routes each node to various kinds of manifold, mixture of spherical, flat, hyperbolic - best matching its local structure. These projections provide intrinsic geometric explanations to the latent space. Inserted into a state-of-the-art ensemble graph transformer, this projector lifts accuracy by up to 3% on four node-classification benchmarks. The ensemble makes sure that both euclidean and non-euclidean features are captured. Explicit, geometry-aware projection thus sharpens predictive power while making graph representations more interpretable.