Assessing the Value of Peer-Produced Information for Exploratory Search
This work addresses the need for better incentives in collaborative tagging systems by providing a quantifiable measure of tag value, though it is incremental as it builds on existing qualitative insights.
The paper tackles the problem of quantifying the value of user-generated tags for exploratory search by proposing an information-theoretical method that measures a tag's ability to reduce search space while retrieving relevant items, with evaluation on real data showing it accurately assigns higher value to tags users consider important.
Tagging is a popular feature that supports several collaborative tasks, including search, as tags produced by one user can help others finding relevant content. However, task performance depends on the existence of 'good' tags. A first step towards creating incentives for users to produce 'good' tags is the quantification of their value in the first place. This work fills this gap by combining qualitative and quantitative research methods. In particular, using contextual interviews, we first determine aspects that influence users' perception of tags' value for exploratory search. Next, we formalize some of the identified aspects and propose an information-theoretical method with provable properties that quantifies the two most important aspects (according to the qualitative analysis) that influence the perception of tag value: the ability of a tag to reduce the search space while retrieving relevant items to the user. The evaluation on real data shows that our method is accurate: tags that users consider more important have higher value than tags users have not expressed interest.