A Qualitative Evaluation of User Preference for Link-based vs. Text-based Recommendations of Wikipedia Articles
This research addresses the gap in understanding user-perceived characteristics of different recommendation approaches for literature systems, though it is incremental as it builds on existing quantitative studies with a qualitative focus.
The study tackled the problem of evaluating user preferences for link-based versus text-based literature recommendation systems, finding that text-based methods scored higher on perceived similarity while link-based methods scored higher on diversity and serendipity, leading to a conclusion that hybrid approaches are most beneficial.
Literature recommendation systems (LRS) assist readers in the discovery of relevant content from the overwhelming amount of literature available. Despite the widespread adoption of LRS, there is a lack of research on the user-perceived recommendation characteristics for fundamentally different approaches to content-based literature recommendation. To complement existing quantitative studies on literature recommendation, we present qualitative study results that report on users' perceptions for two contrasting recommendation classes: (1) link-based recommendation represented by the Co-Citation Proximity (CPA) approach, and (2) text-based recommendation represented by Lucene's MoreLikeThis (MLT) algorithm. The empirical data analyzed in our study with twenty users and a diverse set of 40 Wikipedia articles indicate a noticeable difference between text- and link-based recommendation generation approaches along several key dimensions. The text-based MLT method receives higher satisfaction ratings in terms of user-perceived similarity of recommended articles. In contrast, the CPA approach receives higher satisfaction scores in terms of diversity and serendipity of recommendations. We conclude that users of literature recommendation systems can benefit most from hybrid approaches that combine both link- and text-based approaches, where the user's information needs and preferences should control the weighting for the approaches used. The optimal weighting of multiple approaches used in a hybrid recommendation system is highly dependent on a user's shifting needs.