Tensor-based Sequential Learning via Hankel Matrix Representation for Next Item Recommendations
This work offers a more efficient alternative to transformer models for sequential recommendation tasks, though it is incremental in nature.
The authors tackled the problem of next item recommendation by proposing a lightweight tensor factorization model that mimics self-attention networks, achieving competitive performance compared to neural models.
Self-attentive transformer models have recently been shown to solve the next item recommendation task very efficiently. The learned attention weights capture sequential dynamics in user behavior and generalize well. Motivated by the special structure of learned parameter space, we question if it is possible to mimic it with an alternative and more lightweight approach. We develop a new tensor factorization-based model that ingrains the structural knowledge about sequential data within the learning process. We demonstrate how certain properties of a self-attention network can be reproduced with our approach based on special Hankel matrix representation. The resulting model has a shallow linear architecture and compares competitively to its neural counterpart.