Asking Better Questions -- The Art and Science of Forecasting: A mechanism for truer answers to high-stakes questions
This addresses the need for better AI advancement estimation for organizations and policymakers, but it is incremental as it applies an existing political science tool to a new domain.
The paper tackles the problem of organizations lacking the ability to estimate AI capability advancements, which hinders strategic planning, by exploring forecasting as a tool that improves prediction accuracy and identifies superforecasters who outperform 98% of the population.
Without the ability to estimate and benchmark AI capability advancements, organizations are left to respond to each change reactively, impeding their ability to build viable mid and long-term strategies. This paper explores the recent growth of forecasting, a political science tool that uses explicit assumptions and quantitative estimation that leads to improved prediction accuracy. Done at the collective level, forecasting can identify and verify talent, enable leaders to build better models of AI advancements and improve inputs into design policy. Successful approaches to forecasting and case studies are examined, revealing a subclass of "superforecasters" who outperform 98% of the population and whose insights will be most reliable. Finally, techniques behind successful forecasting are outlined, including Phillip Tetlock's "Ten Commandments." To adapt to a quickly changing technology landscape, designers and policymakers should consider forecasting as a first line of defense.