Learning Feature Interactions with Lorentzian Factorization Machine
This work addresses memory and computational inefficiencies in recommendation and CTR prediction systems, offering a lightweight alternative to deep learning models.
The paper tackles the inefficiency of deep learning methods for feature interaction modeling in recommendation systems and CTR prediction by proposing LorentzFM, a model that embeds interactions in hyperbolic space, reducing parameters by 20% to 80% while achieving comparable or better results than state-of-the-art deep learning methods.
Learning representations for feature interactions to model user behaviors is critical for recommendation system and click-trough rate (CTR) predictions. Recent advances in this area are empowered by deep learning methods which could learn sophisticated feature interactions and achieve the state-of-the-art result in an end-to-end manner. These approaches require large number of training parameters integrated with the low-level representations, and thus are memory and computational inefficient. In this paper, we propose a new model named "LorentzFM" that can learn feature interactions embedded in a hyperbolic space in which the violation of triangle inequality for Lorentz distances is available. To this end, the learned representation is benefited by the peculiar geometric properties of hyperbolic triangles, and result in a significant reduction in the number of parameters (20\% to 80\%) because all the top deep learning layers are not required. With such a lightweight architecture, LorentzFM achieves comparable and even materially better results than the deep learning methods such as DeepFM, xDeepFM and Deep \& Cross in both recommendation and CTR prediction tasks.