GPT as a Baseline for Recommendation Explanation Texts
This work provides an incremental baseline for using LLMs in recommendation systems, targeting users and developers interested in automated explanation generation.
The study investigated whether GPT-generated text explanations for movie recommendations are as helpful as human reviews, finding no significant difference in rankings or quality scores for unseen movies, but participants rated reviews as significantly better for movies they had seen before.
In this work, we establish a baseline potential for how modern model-generated text explanations of movie recommendations may help users, and explore what different components of these text explanations that users like or dislike, especially in contrast to existing human movie reviews. We found that participants gave no significantly different rankings between movies, nor did they give significantly different individual quality scores to reviews of movies that they had never seen before. However, participants did mark reviews as significantly better when they were movies they had seen before. We also explore specific aspects of movie review texts that participants marked as important for each quality. Overall, we establish that modern LLMs are a promising source of recommendation explanations, and we intend on further exploring personalizable text explanations in the future.