Question Rephrasing for Quantifying Uncertainty in Large Language Models: Applications in Molecular Chemistry Tasks
This work addresses uncertainty assessment for users of LLMs in molecular chemistry, but it appears incremental as it builds on existing sampling methods.
The authors tackled the problem of uncertainty quantification in large language models (LLMs) by introducing a Question Rephrasing technique to assess input uncertainty, combined with sampling methods for output uncertainty, and validated it on molecular chemistry tasks such as property and reaction prediction.
Uncertainty quantification enables users to assess the reliability of responses generated by large language models (LLMs). We present a novel Question Rephrasing technique to evaluate the input uncertainty of LLMs, which refers to the uncertainty arising from equivalent variations of the inputs provided to LLMs. This technique is integrated with sampling methods that measure the output uncertainty of LLMs, thereby offering a more comprehensive uncertainty assessment. We validated our approach on property prediction and reaction prediction for molecular chemistry tasks.