Truth-tracking via Approval Voting: Size Matters
This work addresses the challenge of improving truth-tracking in social choice for applications like image annotation, though it is incremental as it builds on existing epistemic models.
The paper tackles the problem of uncovering a hidden ground truth from approval votes by proposing noise models where ballot size indicates reliability, and finds that these models outperform standard approval voting in image annotation datasets, with a Condorcet variant achieving the best performance.
Epistemic social choice aims at unveiling a hidden ground truth given votes, which are interpreted as noisy signals about it. We consider here a simple setting where votes consist of approval ballots: each voter approves a set of alternatives which they believe can possibly be the ground truth. Based on the intuitive idea that more reliable votes contain fewer alternatives, we define several noise models that are approval voting variants of the Mallows model. The likelihood-maximizing alternative is then characterized as the winner of a weighted approval rule, where the weight of a ballot decreases with its cardinality. We have conducted an experiment on three image annotation datasets; they conclude that rules based on our noise model outperform standard approval voting; the best performance is obtained by a variant of the Condorcet noise model.