Effect of Choosing Loss Function when Using T-batching for Representation Learning on Dynamic Networks
This work addresses a specific bottleneck in dynamic network modeling for researchers and practitioners, offering incremental improvements in training performance.
The paper tackled the problem of training dynamic network representation learning models with T-batching by identifying limitations in the original loss function and proposing two alternatives, resulting in over 26.9% improvement in MRR and 11.8% in Recall@10 on a real-world network.
Representation learning methods have revolutionized machine learning on networks by converting discrete network structures into continuous domains. However, dynamic networks that evolve over time pose new challenges. To address this, dynamic representation learning methods have gained attention, offering benefits like reduced learning time and improved accuracy by utilizing temporal information. T-batching is a valuable technique for training dynamic network models that reduces training time while preserving vital conditions for accurate modeling. However, we have identified a limitation in the training loss function used with t-batching. Through mathematical analysis, we propose two alternative loss functions that overcome these issues, resulting in enhanced training performance. We extensively evaluate the proposed loss functions on synthetic and real-world dynamic networks. The results consistently demonstrate superior performance compared to the original loss function. Notably, in a real-world network characterized by diverse user interaction histories, the proposed loss functions achieved more than 26.9% enhancement in Mean Reciprocal Rank (MRR) and more than 11.8% improvement in Recall@10. These findings underscore the efficacy of the proposed loss functions in dynamic network modeling.