Causality-Enhanced Behavior Sequence Modeling in LLMs for Personalized Recommendation
This work addresses a specific bottleneck in personalized recommendation systems for users, offering an incremental improvement over existing LLM-based approaches.
The paper tackles the problem of suboptimal preference modeling in LLM-based recommender systems by proposing a Counterfactual Fine-Tuning method that uses counterfactual reasoning to emphasize user behavior sequences, resulting in improved modeling as demonstrated in experiments on real-world datasets.
Recent advancements in recommender systems have focused on leveraging Large Language Models (LLMs) to improve user preference modeling, yielding promising outcomes. However, current LLM-based approaches struggle to fully leverage user behavior sequences, resulting in suboptimal preference modeling for personalized recommendations. In this study, we propose a novel Counterfactual Fine-Tuning (CFT) method to address this issue by explicitly emphasizing the role of behavior sequences when generating recommendations. Specifically, we employ counterfactual reasoning to identify the causal effects of behavior sequences on model output and introduce a task that directly fits the ground-truth labels based on these effects, achieving the goal of explicit emphasis. Additionally, we develop a token-level weighting mechanism to adjust the emphasis strength for different item tokens, reflecting the diminishing influence of behavior sequences from earlier to later tokens during predicting an item. Extensive experiments on real-world datasets demonstrate that CFT effectively improves behavior sequence modeling. Our codes are available at https://github.com/itsmeyjt/CFT.