AdaGrid: Adaptive Grid Search for Link Prediction Training Objective
This work addresses the challenge of efficient hyperparameter optimization for link prediction in graph neural networks, which is incremental as it builds on existing grid search methods.
The paper tackles the problem of time-consuming and model-dependent hyperparameter tuning for graph neural network training objectives in link prediction by proposing AdaGrid, an adaptive grid search method that dynamically adjusts the edge message ratio during training. The result shows that AdaGrid boosts model performance by up to 1.9% while being nine times more time-efficient than complete grid search.
One of the most important factors that contribute to the success of a machine learning model is a good training objective. Training objective crucially influences the model's performance and generalization capabilities. This paper specifically focuses on graph neural network training objective for link prediction, which has not been explored in the existing literature. Here, the training objective includes, among others, a negative sampling strategy, and various hyperparameters, such as edge message ratio which controls how training edges are used. Commonly, these hyperparameters are fine-tuned by complete grid search, which is very time-consuming and model-dependent. To mitigate these limitations, we propose Adaptive Grid Search (AdaGrid), which dynamically adjusts the edge message ratio during training. It is model agnostic and highly scalable with a fully customizable computational budget. Through extensive experiments, we show that AdaGrid can boost the performance of the models up to $1.9\%$ while being nine times more time-efficient than a complete search. Overall, AdaGrid represents an effective automated algorithm for designing machine learning training objectives.