GraphUniverse: Enabling Systematic Evaluation of Inductive Generalization
This addresses a fundamental gap in graph learning for researchers and practitioners by providing a tool to systematically evaluate and improve model generalization, though it is incremental as it builds on existing synthetic benchmark approaches.
The paper tackles the challenge of evaluating how graph learning models generalize to unseen graphs by introducing GraphUniverse, a framework for generating graph families that enables systematic testing of inductive generalization at scale, revealing that strong transductive performance poorly predicts inductive generalization and that robustness to distribution shift depends on model architecture and initial graph properties.
A fundamental challenge in graph learning is understanding how models generalize to new, unseen graphs. While synthetic benchmarks offer controlled settings for analysis, existing approaches are confined to single-graph, transductive settings where models train and test on the same graph structure. Addressing this gap, we introduce GraphUniverse, a framework for generating entire families of graphs to enable the first systematic evaluation of inductive generalization at scale. Our core innovation is the generation of graphs with persistent semantic communities, ensuring conceptual consistency while allowing fine-grained control over structural properties like homophily and degree distributions. This enables crucial but underexplored robustness tests, such as performance under controlled distribution shifts. Benchmarking a wide range of architectures -- from GNNs to graph transformers and topological architectures -- reveals that strong transductive performance is a poor predictor of inductive generalization. Furthermore, we find that robustness to distribution shift is highly sensitive not only to model architecture choice but also to the initial graph regime (e.g., high vs. low homophily). Beyond benchmarking, GraphUniverse's flexibility and scalability can facilitate the development of robust and truly generalizable architectures -- including next-generation graph foundation models. An interactive demo is available at https://graphuniverse.streamlit.app.