Scalable Heterogeneous Graph Learning via Heterogeneous-aware Orthogonal Prototype Experts
This addresses a decoding bottleneck in heterogeneous graph learning, offering a practical improvement for researchers and practitioners in graph-based AI applications.
The paper tackles the Linear Projection Bottleneck in Heterogeneous Graph Neural Networks, where a single shared linear head fails to handle contextual diversity and long-tail shifts, by proposing HOPE, a plug-and-play framework that uses prototype-based routing and orthogonalization to improve prediction. Experiments on four real datasets show consistent gains across state-of-the-art backbones with minimal overhead.
Heterogeneous Graph Neural Networks(HGNNs) have advanced mainly through better encoders, yet their decoding/projection stage still relies on a single shared linear head, assuming it can map rich node embeddings to labels. We call this the Linear Projection Bottleneck: in heterogeneous graphs, contextual diversity and long-tail shifts make a global head miss fine semantics, overfit hub nodes, and underserve tail nodes. While Mixture-of-Experts(MoE) could help, naively applying it clashes with structural imbalance and risks expert collapse. We propose a Heterogeneous-aware Orthogonal Prototype Experts framework named HOPE, a plug-and-play replacement for the standard prediction head. HOPE uses learnable prototype-based routing to assign instances to experts by similarity, letting expert usage follow the natural long-tail distribution, and adds expert orthogonalization to encourage diversity and prevent collapse. Experiments on four real datasets show consistent gains across SOTA HGNN backbones with minimal overhead.