Are topic-specific search term, journal name and author name recommendations relevant for researchers?
This addresses the problem of improving search efficiency for researchers in digital libraries, but it is incremental as it applies existing bibliometric-enhanced IR methods to a specific domain.
The study evaluated the relevance of automatically generated search term, journal name, and author name recommendations for social science researchers, finding high relevance with average precision scores of 0.75, 0.74, and 0.73, respectively, and noting variations across topics and researcher types.
In this paper we describe a case study where researchers in the social sciences (n=19) assess topical relevance for controlled search terms, journal names and author names which have been compiled automatically by bibliometric-enhanced information retrieval (IR) services. We call these bibliometric-enhanced IR services Search Term Recommender (STR), Journal Name Recommender (JNR) and Author Name Recommender (ANR) in this paper. The researchers in our study (practitioners, PhD students and postdocs) were asked to assess the top n pre-processed recommendations from each recommender for specific research topics which have been named by them in an interview before the experiment. Our results show clearly that the presented search term, journal name and author name recommendations are highly relevant to the researchers' topic and can easily be integrated for search in Digital Libraries. The average precision for top ranked recommendations is 0.75 for author names, 0.74 for search terms and 0.73 for journal names. The relevance distribution differs largely across topics and researcher types. Practitioners seem to favor author name recommendations while postdocs have rated author name recommendations the lowest. In the experiment the small postdoc group (n=3) favor journal name recommendations.