Scott Cunningham

Van Pham, Scott Cunningham

This study investigates whether OpenAI's ChatGPT-3.5 and ChatGPT-4 can forecast future events. To evaluate the accuracy of the predictions, we take advantage of the fact that the training data at the time of our experiments (mid 2023) stopped at September 2021, and ask about events that happened in 2022. We employed two prompting strategies: direct prediction and what we call future narratives which ask ChatGPT to tell fictional stories set in the future with characters retelling events that happened in the past, but after ChatGPT's training data had been collected. We prompted ChatGPT to engage in storytelling, particularly within economic contexts. After analyzing 100 trials, we find that future narrative prompts significantly enhanced ChatGPT-4's forecasting accuracy. This was especially evident in its predictions of major Academy Award winners as well as economic trends, the latter inferred from scenarios where the model impersonated public figures like the Federal Reserve Chair, Jerome Powell. As a falsification exercise, we repeated our experiments in May 2024 at which time the models included more recent training data. ChatGPT-4's accuracy significantly improved when the training window included the events being prompted for, achieving 100% accuracy in many instances. The poorer accuracy for events outside of the training window suggests that in the 2023 prediction experiments, ChatGPT-4 was forming predictions based solely on its training data. Narrative prompting also consistently outperformed direct prompting. These findings indicate that narrative prompts leverage the models' capacity for hallucinatory narrative construction, facilitating more effective data synthesis and extrapolation than straightforward predictions. Our research reveals new aspects of LLMs' predictive capabilities and suggests potential future applications in analytical contexts.

Trivik Verma, Mikhail Sirenko, Itto Kornecki et al.

With people constantly migrating to different urban areas, our mobility needs for work, services and leisure are transforming rapidly. The changing urban demographics pose several challenges for the efficient management of transit services. To forecast transit demand, planners often resort to sociological investigations or modelling that are either difficult to obtain, inaccurate or outdated. How can we then estimate the variegated demand for mobility? We propose a simple method to identify the spatiotemporal demand for public transit in a city. Using a Gaussian mixture model, we decompose empirical ridership data into a set of temporal demand profiles representative of ridership over any given day. A case of approximately 4.6 million daily transit traces from the Greater London region reveals distinct demand profiles. We find that a weighted mixture of these profiles can generate any station traffic remarkably well, uncovering spatially concentric clusters of mobility needs. Our method of analysing the spatiotemporal geography of a city can be extended to other urban regions with different modes of public transit.