MTRec: Learning to Align with User Preferences via Mental Reward Models
This addresses the challenge of inaccurate user feedback in recommender systems, particularly for platforms like short video services, though it is an incremental improvement over existing methods.
The paper tackles the problem of recommendation models being misled by implicit feedback that does not reflect real user preferences, proposing MTRec to align with user satisfaction, resulting in a 7% increase in average user viewing time on an industrial platform.
Recommendation models are predominantly trained using implicit user feedback, since explicit feedback is often costly to obtain. However, implicit feedback, such as clicks, does not always reflect users' real preferences. For example, a user might click on a news article because of its attractive headline, but end up feeling uncomfortable after reading the content. In the absence of explicit feedback, such erroneous implicit signals may severely mislead recommender systems. In this paper, we propose MTRec, a novel sequential recommendation framework designed to align with real user preferences by uncovering their internal satisfaction on recommended items. Specifically, we introduce a mental reward model to quantify user satisfaction and propose a distributional inverse reinforcement learning approach to learn it. The learned mental reward model is then used to guide recommendation models to better align with users' real preferences. Our experiments show that MTRec brings significant improvements to a variety of recommendation models. We also deploy MTRec on an industrial short video platform and observe a 7 percent increase in average user viewing time.