GraphBench: Next-generation graph learning benchmarking
This addresses the need for reproducible and consistent benchmarking in graph learning, which is crucial for researchers and practitioners in fields like molecular property prediction and chip design, though it is incremental as it builds on existing benchmarking practices.
The authors tackled the problem of fragmented benchmarking in graph machine learning by introducing GraphBench, a comprehensive benchmarking suite that spans diverse domains and prediction tasks with standardized evaluation protocols, and they established reference performance by benchmarking it with message-passing neural networks and graph transformer models.
Machine learning on graphs has recently achieved impressive progress in various domains, including molecular property prediction and chip design. However, benchmarking practices remain fragmented, often relying on narrow, task-specific datasets and inconsistent evaluation protocols, which hampers reproducibility and broader progress. To address this, we introduce GraphBench, a comprehensive benchmarking suite that spans diverse domains and prediction tasks, including node-level, edge-level, graph-level, and generative settings. GraphBench provides standardized evaluation protocols -- with consistent dataset splits and performance metrics that account for out-of-distribution generalization -- as well as a unified hyperparameter tuning framework. Additionally, we benchmark GraphBench using message-passing neural networks and graph transformer models, providing principled baselines and establishing a reference performance. See www.graphbench.io for further details.