Hedging and Approximate Truthfulness in Traditional Forecasting Competitions
This addresses incentive problems in forecasting competitions, providing formal insights for organizers and participants, though it is incremental as it builds on known issues.
The paper analyzes the traditional forecasting competition mechanism, showing that the folklore belief of long-run truthfulness is false as forecasters can hedge to increase win probability, but also demonstrates that approximate truthfulness holds for two contestants under sufficient uncertainty about opponent quality and event outcomes.
In forecasting competitions, the traditional mechanism scores the predictions of each contestant against the outcome of each event, and the contestant with the highest total score wins. While it is well-known that this traditional mechanism can suffer from incentive issues, it is folklore that contestants will still be roughly truthful as the number of events grows. Yet thus far the literature lacks a formal analysis of this traditional mechanism. This paper gives the first such analysis. We first demonstrate that the ''long-run truthfulness'' folklore is false: even for arbitrary numbers of events, the best forecaster can have an incentive to hedge, reporting more moderate beliefs to increase their win probability. On the positive side, however, we show that two contestants will be approximately truthful when they have sufficient uncertainty over the relative quality of their opponent and the outcomes of the events, a case which may arise in practice.