Revisiting Embeddings for Graph Neural Networks
This work addresses the impact of embeddings on GNN performance, highlighting a bias in existing datasets, which is incremental as it revisits and analyzes current practices without introducing new methods.
The paper investigates how different embedding techniques affect Graph Neural Network (GNN) accuracy, finding that GNN performance depends on embedding style and that current datasets favor bag-of-words embeddings, leading to GNNs optimized for such vectors.
Current graph representation learning techniques use Graph Neural Networks (GNNs) to extract features from dataset embeddings. In this work, we examine the quality of these embeddings and assess how changing them can affect the accuracy of GNNs. We explore different embedding extraction techniques for both images and texts; and find that the performance of different GNN architectures is dependent on the embedding style used. We see a prevalence of bag of words (BoW) embeddings and text classification tasks in available graph datasets. Given the impact embeddings has on GNN performance. this leads to a phenomenon that GNNs being optimised for BoW vectors.