From Features to Graphs: Exploring Graph Structures and Pairwise Interactions via GNNs
This work addresses the challenge of optimizing feature graph structures for GNNs in predictive modeling, offering incremental improvements in performance and interpretability for researchers and practitioners using GNNs.
The paper tackled the problem of designing feature graphs for Graph Neural Networks (GNNs) to model pairwise feature interactions, finding that sparse graphs with only necessary interaction edges improve performance and interpretability, as shown through experiments on synthesized datasets and theoretical support from the Minimum Description Length principle.
Feature interaction is crucial in predictive machine learning models, as it captures the relationships between features that influence model performance. In this work, we focus on pairwise interactions and investigate their importance in constructing feature graphs for Graph Neural Networks (GNNs). We leverage existing GNN models and tools to explore the relationship between feature graph structures and their effectiveness in modeling interactions. Through experiments on synthesized datasets, we uncover that edges between interacting features are important for enabling GNNs to model feature interactions effectively. We also observe that including non-interaction edges can act as noise, degrading model performance. Furthermore, we provide theoretical support for sparse feature graph selection using the Minimum Description Length (MDL) principle. We prove that feature graphs retaining only necessary interaction edges yield a more efficient and interpretable representation than complete graphs, aligning with Occam's Razor. Our findings offer both theoretical insights and practical guidelines for designing feature graphs that improve the performance and interpretability of GNN models.