Multi-relational Learning Using Weighted Tensor Decomposition with Modular Loss
This work addresses multi-relational learning for applications like recommendation systems or knowledge graphs, but it is incremental as it builds on existing tensor decomposition techniques.
The authors tackled the problem of predicting missing entries in sparse multi-relational data by proposing a weighted tensor decomposition framework with modular loss functions, resulting in significant improvements in accuracy and scalability, with efficiency gains potentially up to an order of magnitude over unweighted methods.
We propose a modular framework for multi-relational learning via tensor decomposition. In our learning setting, the training data contains multiple types of relationships among a set of objects, which we represent by a sparse three-mode tensor. The goal is to predict the values of the missing entries. To do so, we model each relationship as a function of a linear combination of latent factors. We learn this latent representation by computing a low-rank tensor decomposition, using quasi-Newton optimization of a weighted objective function. Sparsity in the observed data is captured by the weighted objective, leading to improved accuracy when training data is limited. Exploiting sparsity also improves efficiency, potentially up to an order of magnitude over unweighted approaches. In addition, our framework accommodates arbitrary combinations of smooth, task-specific loss functions, making it better suited for learning different types of relations. For the typical cases of real-valued functions and binary relations, we propose several loss functions and derive the associated parameter gradients. We evaluate our method on synthetic and real data, showing significant improvements in both accuracy and scalability over related factorization techniques.