When Heterophily Meets Heterogeneity: Challenges and a New Large-Scale Graph Benchmark
This addresses a gap in graph learning for researchers by providing a standardized benchmark to evaluate models on real-world graphs with combined heterophily and heterogeneity, though it is incremental as it builds on existing benchmarking efforts.
The paper tackles the lack of benchmarks for graphs with both heterogeneity and heterophily by introducing H2GB, a large-scale node-classification benchmark with 9 datasets and 28 baseline models, showing that current methods struggle on such graphs and presenting a new model, H2G-former, that excels in this setting.
Graph mining has become crucial in fields such as social science, finance, and cybersecurity. Many large-scale real-world networks exhibit both heterogeneity, where multiple node and edge types exist in the graph, and heterophily, where connected nodes may have dissimilar labels and attributes. However, existing benchmarks primarily focus on either heterophilic homogeneous graphs or homophilic heterogeneous graphs, leaving a significant gap in understanding how models perform on graphs with both heterogeneity and heterophily. To bridge this gap, we introduce H2GB, a large-scale node-classification graph benchmark that brings together the complexities of both the heterophily and heterogeneity properties of real-world graphs. H2GB encompasses 9 real-world datasets spanning 5 diverse domains, 28 baseline models, and a unified benchmarking library with a standardized data loader, evaluator, unified modeling framework, and an extensible framework for reproducibility. We establish a standardized workflow supporting both model selection and development, enabling researchers to easily benchmark graph learning methods. Extensive experiments across 28 baselines reveal that current methods struggle with heterophilic and heterogeneous graphs, underscoring the need for improved approaches. Finally, we present a new variant of the model, H2G-former, developed following our standardized workflow, that excels at this challenging benchmark. Both the benchmark and the framework are publicly available at Github and PyPI, with documentation hosted at https://junhongmit.github.io/H2GB.