RaFM: Rank-Aware Factorization Machines
This work addresses a limitation in factorization machines for machine learning practitioners, offering an incremental improvement with potential industrial applications.
The paper tackles the problem of factorization machines using a fixed rank for all features by proposing a Rank-Aware FM model that uses embeddings with different ranks, achieving better performance on real-world datasets with varying feature frequencies and proving efficiency comparable to or better than standard FMs.
Factorization machines (FM) are a popular model class to learn pairwise interactions by a low-rank approximation. Different from existing FM-based approaches which use a fixed rank for all features, this paper proposes a Rank-Aware FM (RaFM) model which adopts pairwise interactions from embeddings with different ranks. The proposed model achieves a better performance on real-world datasets where different features have significantly varying frequencies of occurrences. Moreover, we prove that the RaFM model can be stored, evaluated, and trained as efficiently as one single FM, and under some reasonable conditions it can be even significantly more efficient than FM. RaFM improves the performance of FMs in both regression tasks and classification tasks while incurring less computational burden, therefore also has attractive potential in industrial applications.