Inducing Artificial Uncertainty in Language Models
This work addresses the growing difficulty of obtaining challenging data for training uncertainty quantification in large language models, which is important for safety-critical applications.
The authors introduce the problem of inducing artificial uncertainty in language models to train uncertainty quantification methods when challenging data is unavailable. They show that probes trained on artificial uncertainty outperform those trained without it, achieving higher calibration on hard data with minimal performance loss on easy data.
In safety-critical applications, language models should be able to characterize their uncertainty with meaningful probabilities. Many uncertainty quantification approaches require supervised data; however, finding suitable unseen challenging data is increasingly difficult for large language models trained on vast amounts of scraped data. If the model is consistently (and correctly) confident in its predictions, the uncertainty quantification method may consistently overestimate confidence on new and unfamiliar data. Finding data which exhibits enough uncertainty to train supervised uncertainty quantification methods for high-performance models may therefore be challenging, and will increase in difficulty as LLMs saturate datasets. To address this issue, we first introduce the problem of inducing artificial uncertainty in language models, then investigate methods of inducing artificial uncertainty on trivially easy data in the absence of challenging data at training time. We use probes trained to recognize artificial uncertainty on the original model, and find that these probes trained on artificial uncertainty outperform probes trained without artificial uncertainty in recognizing real uncertainty, achieving notably higher calibration on hard data with minimal loss of performance on easy data.