Predicting Pairwise Relations with Neural Similarity Encoders
This addresses a bottleneck in machine learning applications like recommender systems by providing a neural network-based solution for matrix factorization and similarity prediction, though it appears incremental as an extension of neural SVD methods.
The paper tackles the problem of efficiently factorizing large matrices with missing values and learning mappings from feature vectors to pairwise relations, introducing the Similarity Encoder (SimEc) architecture that enables out-of-sample predictions and handles non-metric similarities.
Matrix factorization is at the heart of many machine learning algorithms, for example, dimensionality reduction (e.g. kernel PCA) or recommender systems relying on collaborative filtering. Understanding a singular value decomposition (SVD) of a matrix as a neural network optimization problem enables us to decompose large matrices efficiently while dealing naturally with missing values in the given matrix. But most importantly, it allows us to learn the connection between data points' feature vectors and the matrix containing information about their pairwise relations. In this paper we introduce a novel neural network architecture termed Similarity Encoder (SimEc), which is designed to simultaneously factorize a given target matrix while also learning the mapping to project the data points' feature vectors into a similarity preserving embedding space. This makes it possible to, for example, easily compute out-of-sample solutions for new data points. Additionally, we demonstrate that SimEc can preserve non-metric similarities and even predict multiple pairwise relations between data points at once.