One4all User Representation for Recommender Systems in E-commerce
This work addresses efficient user representation for e-commerce tasks like recommendation, though it is incremental as it builds on existing pre-training methods.
The paper tackled the problem of creating general-purpose user representations for e-commerce by pre-training ShopperBERT on 0.8B user behaviors, resulting in embeddings that outperformed task-specific models in five out of six downstream tasks.
General-purpose representation learning through large-scale pre-training has shown promising results in the various machine learning fields. For an e-commerce domain, the objective of general-purpose, i.e., one for all, representations would be efficient applications for extensive downstream tasks such as user profiling, targeting, and recommendation tasks. In this paper, we systematically compare the generalizability of two learning strategies, i.e., transfer learning through the proposed model, ShopperBERT, vs. learning from scratch. ShopperBERT learns nine pretext tasks with 79.2M parameters from 0.8B user behaviors collected over two years to produce user embeddings. As a result, the MLPs that employ our embedding method outperform more complex models trained from scratch for five out of six tasks. Specifically, the pre-trained embeddings have superiority over the task-specific supervised features and the strong baselines, which learn the auxiliary dataset for the cold-start problem. We also show the computational efficiency and embedding visualization of the pre-trained features.