Objective Social Choice: Using Auxiliary Information to Improve Voting Outcomes
This work addresses a foundational issue in social choice and ensembling for applications like policy-making and machine learning, though it appears incremental by building on existing models with auxiliary data.
The paper tackles the problem of aggregating noisy votes from diverse sources to infer an objective ground truth by relaxing the unrealistic i.i.d. assumption and using auxiliary information about noise models. It proposes new aggregation rules, including maximum likelihood and neural network approaches, which empirically outperform naive baselines.
How should one combine noisy information from diverse sources to make an inference about an objective ground truth? This frequently recurring, normative question lies at the core of statistics, machine learning, policy-making, and everyday life. It has been called "combining forecasts", "meta-analysis", "ensembling", and the "MLE approach to voting", among other names. Past studies typically assume that noisy votes are identically and independently distributed (i.i.d.), but this assumption is often unrealistic. Instead, we assume that votes are independent but not necessarily identically distributed and that our ensembling algorithm has access to certain auxiliary information related to the underlying model governing the noise in each vote. In our present work, we: (1) define our problem and argue that it reflects common and socially relevant real world scenarios, (2) propose a multi-arm bandit noise model and count-based auxiliary information set, (3) derive maximum likelihood aggregation rules for ranked and cardinal votes under our noise model, (4) propose, alternatively, to learn an aggregation rule using an order-invariant neural network, and (5) empirically compare our rules to common voting rules and naive experience-weighted modifications. We find that our rules successfully use auxiliary information to outperform the naive baselines.