LLM Generated Distribution-Based Prediction of US Electoral Results, Part I
This method addresses the need for increased transparency and reliability in LLM-based predictions across various domains, though it appears incremental as it builds on existing approaches like silicon sampling.
The paper tackles the problem of using LLMs as predictive tools by introducing distribution-based prediction, which interprets token probabilities as distributions to analyze algorithmic fidelity, bias, and prompt noise, demonstrating its application in predicting US presidential election results.
This paper introduces distribution-based prediction, a novel approach to using Large Language Models (LLMs) as predictive tools by interpreting output token probabilities as distributions representing the models' learned representation of the world. This distribution-based nature offers an alternative perspective for analyzing algorithmic fidelity, complementing the approach used in silicon sampling. We demonstrate the use of distribution-based prediction in the context of recent United States presidential election, showing that this method can be used to determine task specific bias, prompt noise, and algorithmic fidelity. This approach has significant implications for assessing the reliability and increasing transparency of LLM-based predictions across various domains.