Mathematical Modeling of Competitive Group Recommendation Systems with Application to Peer Review Systems
This work addresses the challenge of improving recommendation accuracy in peer review systems, which is crucial for scientific communities, though it is incremental as it builds on existing models for group recommendations.
The paper tackles the problem of designing accurate competitive group recommendation systems, specifically for peer review, by developing a mathematical model and analyzing factors like number of reviews and aggregation policies; it finds that three reviews suffice for medium-tier conferences, while prestigious ones need at least seven, and proposes a heterogeneous strategy that improves accuracy by up to 30% with equal or less workload.
In this paper, we present a mathematical model to capture various factors which may influence the accuracy of a competitive group recommendation system. We apply this model to peer review systems, i.e., conference or research grants review, which is an essential component in our scientific community. We explore number of important questions, i.e., how will the number of reviews per paper affect the accuracy of the overall recommendation? Will the score aggregation policy influence the final recommendation? How reviewers' preference may affect the accuracy of the final recommendation? To answer these important questions, we formally analyze our model. Through this analysis, we obtain the insight on how to design a randomized algorithm which is both computationally efficient and asymptotically accurate in evaluating the accuracy of a competitive group recommendation system. We obtain number of interesting observations: i.e., for a medium tier conference, three reviews per paper is sufficient for a high accuracy recommendation. For prestigious conferences, one may need at least seven reviews per paper to achieve high accuracy. We also propose a heterogeneous review strategy which requires equal or less reviewing workload, but can improve over a homogeneous review strategy in recommendation accuracy by as much as 30% . We believe our models and methodology are important building blocks to study competitive group recommendation systems.