UPRec: User-Aware Pre-training for Recommender Systems
This addresses personalized recommendation for users by integrating user information into pre-training, though it is incremental as it builds on existing pre-training methods.
The paper tackles the data sparsity problem in sequential recommendation by enhancing pre-trained models with heterogeneous user information, such as attributes and social graphs, through self-supervised tasks, resulting in more appropriate recommendations as demonstrated on large-scale datasets.
Existing sequential recommendation methods rely on large amounts of training data and usually suffer from the data sparsity problem. To tackle this, the pre-training mechanism has been widely adopted, which attempts to leverage large-scale data to perform self-supervised learning and transfer the pre-trained parameters to downstream tasks. However, previous pre-trained models for recommendation focus on leverage universal sequence patterns from user behaviour sequences and item information, whereas ignore capturing personalized interests with the heterogeneous user information, which has been shown effective in contributing to personalized recommendation. In this paper, we propose a method to enhance pre-trained models with heterogeneous user information, called User-aware Pre-training for Recommendation (UPRec). Specifically, UPRec leverages the user attributes andstructured social graphs to construct self-supervised objectives in the pre-training stage and proposes two user-aware pre-training tasks. Comprehensive experimental results on several real-world large-scale recommendation datasets demonstrate that UPRec can effectively integrate user information into pre-trained models and thus provide more appropriate recommendations for users.