Efficient Pairwise Learning Using Kernel Ridge Regression: an Exact Two-Step Method
This work addresses pairwise learning problems like matrix completion and collaborative filtering, providing a practical tool for large-scale applications, though it is incremental as it builds on existing kernel methods.
The paper tackles the problem of pairwise learning by analyzing a two-step kernel-based method as an alternative to Kronecker-based approaches, showing it offers efficient training and validation on large datasets with extensive experimental validation.
Pairwise learning or dyadic prediction concerns the prediction of properties for pairs of objects. It can be seen as an umbrella covering various machine learning problems such as matrix completion, collaborative filtering, multi-task learning, transfer learning, network prediction and zero-shot learning. In this work we analyze kernel-based methods for pairwise learning, with a particular focus on a recently-suggested two-step method. We show that this method offers an appealing alternative for commonly-applied Kronecker-based methods that model dyads by means of pairwise feature representations and pairwise kernels. In a series of theoretical results, we establish correspondences between the two types of methods in terms of linear algebra and spectral filtering, and we analyze their statistical consistency. In addition, the two-step method allows us to establish novel algorithmic shortcuts for efficient training and validation on very large datasets. Putting those properties together, we believe that this simple, yet powerful method can become a standard tool for many problems. Extensive experimental results for a range of practical settings are reported.