Making Paper Reviewing Robust to Bid Manipulation Attacks
This work addresses the problem of ensuring fair and unbiased paper review assignments for conference organizers and the academic community, which is increasingly important given rising submission numbers and potential for manipulation.
This paper investigates bid manipulation attacks in computer science conference paper assignments, finding they can compromise review integrity. The authors propose a novel bidding and assignment approach that demonstrates robustness against such attacks, even under collusion and full system knowledge, while maintaining comparable assignment quality.
Most computer science conferences rely on paper bidding to assign reviewers to papers. Although paper bidding enables high-quality assignments in days of unprecedented submission numbers, it also opens the door for dishonest reviewers to adversarially influence paper reviewing assignments. Anecdotal evidence suggests that some reviewers bid on papers by "friends" or colluding authors, even though these papers are outside their area of expertise, and recommend them for acceptance without considering the merit of the work. In this paper, we study the efficacy of such bid manipulation attacks and find that, indeed, they can jeopardize the integrity of the review process. We develop a novel approach for paper bidding and assignment that is much more robust against such attacks. We show empirically that our approach provides robustness even when dishonest reviewers collude, have full knowledge of the assignment system's internal workings, and have access to the system's inputs. In addition to being more robust, the quality of our paper review assignments is comparable to that of current, non-robust assignment approaches.