Estimating predictive uncertainty for rumour verification models
This addresses the challenge of unreliable automated fact-checking for online rumours, though it is incremental as it builds on existing NLP models.
The paper tackles the problem of improving rumour verification models by incorporating predictive uncertainty estimates, which can filter out likely erroneous predictions for human review and interpret model performance over time.
The inability to correctly resolve rumours circulating online can have harmful real-world consequences. We present a method for incorporating model and data uncertainty estimates into natural language processing models for automatic rumour verification. We show that these estimates can be used to filter out model predictions likely to be erroneous, so that these difficult instances can be prioritised by a human fact-checker. We propose two methods for uncertainty-based instance rejection, supervised and unsupervised. We also show how uncertainty estimates can be used to interpret model performance as a rumour unfolds.