Exponential Machines
This addresses the challenge of high-order feature interactions for domains like recommender systems, though it is incremental as it builds on existing factorization methods.
The paper tackles the problem of modeling all feature interactions in machine learning by introducing Exponential Machines (ExM), which uses a Tensor Train factorization to handle exponentially large parameter tensors and achieves state-of-the-art performance on synthetic data and competitive results on the MovieLens 100K dataset.
Modeling interactions between features improves the performance of machine learning solutions in many domains (e.g. recommender systems or sentiment analysis). In this paper, we introduce Exponential Machines (ExM), a predictor that models all interactions of every order. The key idea is to represent an exponentially large tensor of parameters in a factorized format called Tensor Train (TT). The Tensor Train format regularizes the model and lets you control the number of underlying parameters. To train the model, we develop a stochastic Riemannian optimization procedure, which allows us to fit tensors with 2^160 entries. We show that the model achieves state-of-the-art performance on synthetic data with high-order interactions and that it works on par with high-order factorization machines on a recommender system dataset MovieLens 100K.