Comprehensive Analysis of Negative Sampling in Knowledge Graph Representation Learning
This work addresses hyperparameter tuning challenges for researchers and practitioners in knowledge graph representation learning, offering theoretical insights to reduce computational costs, though it is incremental as it builds on existing methods.
The paper tackled the problem of hyperparameter tuning in negative sampling loss for knowledge graph embedding learning by providing a theoretical analysis to guide better selection of margin terms and negative sample numbers, showing that scoring methods with restricted value ranges require different adjustments than those without, and empirical results on datasets like FB15k-237, WN18RR, and YAGO3-10 confirmed these findings.
Negative sampling (NS) loss plays an important role in learning knowledge graph embedding (KGE) to handle a huge number of entities. However, the performance of KGE degrades without hyperparameters such as the margin term and number of negative samples in NS loss being appropriately selected. Currently, empirical hyperparameter tuning addresses this problem at the cost of computational time. To solve this problem, we theoretically analyzed NS loss to assist hyperparameter tuning and understand the better use of the NS loss in KGE learning. Our theoretical analysis showed that scoring methods with restricted value ranges, such as TransE and RotatE, require appropriate adjustment of the margin term or the number of negative samples different from those without restricted value ranges, such as RESCAL, ComplEx, and DistMult. We also propose subsampling methods specialized for the NS loss in KGE studied from a theoretical aspect. Our empirical analysis on the FB15k-237, WN18RR, and YAGO3-10 datasets showed that the results of actually trained models agree with our theoretical findings.