A Process for the Evaluation of Node Embedding Methods in the Context of Node Classification
This provides a reproducible framework for researchers and practitioners to compare node embedding methods, though it is incremental as it focuses on evaluation rather than new methods.
The paper tackles the lack of standardized evaluation for node embedding methods in node classification by developing a fair and objective process, applying it to four popular methods and finding that good performance can be achieved with lower-dimensional embeddings and minimal hyperparameter tuning.
Node embedding methods find latent lower-dimensional representations which are used as features in machine learning models. In the last few years, these methods have become extremely popular as a replacement for manual feature engineering. Since authors use various approaches for the evaluation of node embedding methods, existing studies can rarely be efficiently and accurately compared. We address this issue by developing a process for a fair and objective evaluation of node embedding procedures w.r.t. node classification. This process supports researchers and practitioners to compare new and existing methods in a reproducible way. We apply this process to four popular node embedding methods and make valuable observations. With an appropriate combination of hyperparameters, good performance can be achieved even with embeddings of lower dimensions, which is positive for the run times of the downstream machine learning task and the embedding algorithm. Multiple hyperparameter combinations yield similar performance. Thus, no extensive, time-consuming search is required to achieve reasonable performance in most cases.