Comments
This is cross-posted/adapted from FRI's substack announcement. I'm collaborating with FRI (in alphabetical order, Ezra Karger, Nick Merrill, Phil Tetlock, and Bridget Williams) and Jason Abaluck on this project. Extra comments are my own opinions and do not necessarily reflect those of the group.
Forecasts about the likelihood of catastrophic risks from AI vary wildly. Dario Amodei has estimated a 25% chance of human extinction within the next few decades. Experts, meanwhile, put the chance of an AI-related catastrophe at 0.3% by 2030 and 2% by 2050. Understanding the level of risk and taking appropriate action is one of the most consequential challenges facing humans today.
Forecasts from frontier AI models could be an important input into this debate. The best models now approach superforecaster levels of accuracy, and models are rapidly becoming more capable. To take advantage of these capabilities, we're introducing the Automated AI Risk Outlook (AIRO)—a fully automated dashboard of catastrophic risk outcomes as forecast by frontier AI models. AIRO enables us to track in close to real time how the likelihood of catastrophic outcomes changes, and how these risk forecasts relate to advances in model capability. The accompanying white paper also presents preliminary forecasts conditional on policy scenarios.
The AIRO dashboard currently puts the likelihood of an AI-related catastrophe that kills at least 10% of the population at 0.47% by the end of 2030 and 6% by the end of 2050. In addition to this headline forecast, AIRO tracks risks from AI-specific catastrophes and incidents, biorisk, cyberrisk, and misalignment risk.
A key feature of AIRO is that forecasts are repeated over time, so one can see how they change or if they spike. LLMs can forecast many outcomes across many time horizons and severity thresholds without growing tired, allowing us to elicit many more forecasts than is possible with human forecasters.
Why trust LLM risk forecasts?
AI forecasting abilities have improved alongside other AI capabilities. In July 2026, an AI model reached parity with superforecasters on FRI’s forecasting benchmark, ForecastBench. This gives some basis for believing that AI forecasting may now be state of the art.
The forecasts shown on AIRO are different from those evaluated in ForecastBench in two important ways. First, they concern very rare events. It is difficult to measure accuracy on forecasting rare events given that they, by definition, happen so rarely. However, unlike human forecasters, we can ask AI models to make thousands of predictions on events in a simulated world environment. When we do this, we find that more capable models—as assessed by their score on the ECI—are more accurate at forecasting rare events.
Second, AIRO includes many conditional forecasting questions. The dashboard includes forecasts conditional on model capabilities; the white paper also explores preliminary policy scenarios. We can again test performance on this type of conditional forecasting in a simulated world environment, and again we find that accuracy correlates with general capabilities.
There are risks associated with relying on automated AI forecasters of catastrophic risk. For example, AI forecasts may be influenced by strategic behavior: a highly capable but misaligned model trying to evade human suspicion may lower its estimates to avoid causing alarm. This “sandbagging” behavior may also arise if companies, concerned about regulation or other threats to profit, intentionally post-train models to underplay the risk of AI-caused harms.
How AIRO works
Our forecasting questions are drawn from earlier FRI work, allowing us to compare forecasts from the LLM ensemble with those from experts and superforecasters. We elicit forecasts from the four highest-scoring eligible models on the ECI, keeping one model per family.
Each model answers all questions—across time horizons and severity thresholds—in one forecasting session. Each model conducts its own research via an agentic harness. A prompt asks it to investigate recent developments, expert reports, and published estimates for each cause of catastrophe, and use what it learns to formulate follow-up queries. The model chooses searches, receives results, and decides what to investigate next. All models receive the forecasting date and questions and use the web-search and page-reading tools supplied through Tavily. Further details are provided on AIRO's website.
Some interesting stats
In order to gauge how forecasts vary with severity, some forecasting questions ask models to predict damages either in terms of money or deaths. For example, 100 deaths would be equivalent to economic losses of $220 million, using estimates of the value of a statistical life. Results were interesting, because you can see that for low levels of incident "severity", risks from cyber incidents dominate, but at high levels, misalignment risks are more likely - there is a crossing point:
Biorisks were assessed as relatively less likely than AI-related risks, except in the case of a severe incident in the short term:
Limitations, benefits, and what's next
There is a lot of uncertainty underlying the forecasts. It could be harmful if people overly latched onto them. The forecasts also do not show the positive effects that AI can have - a fuller treatment might use conditional forecasts to assess the pros and cons of different policy options.
Forecasts of AI impacts might be particularly valuable if frontier labs were interested in participating in some way to make their own forecasts and see how they evolve. The labs, after all, have private information, including about unreleased models. The question is whether they would have the incentive to. As an economist, I take incentives very seriously. So, I see many ways this could go where the forecasts would stop being informative - but I also think there is real value to AI companies in showing risks and rewards transparently. Careful forecasts, validated over a long period, are something that could help them build trust, and that could be something that they would benefit from even in a self-serving way.
Personally, I am glad this dashboard now exists because I believe that despite noise there is some signal here. Past research on forecasting indicates that even when forecasts get the absolute level of an outcome wrong, they are often good at ranking different options or observing time trends, and I believe the AI forecasts are probably already better than human forecasts and likely to get better.
AIRO is still a work in progress. We plan to keep validating and extending our dashboard by refining ensemble forecasts and developing a larger set of intermediate questions related to catastrophic risk outcomes. These intermediate forecasts will serve as validation tools and provide inputs that future models can use to inform their risk assessments. We'll also continue to test conditional forecasts to better understand the costs and benefits of proposed policies to mitigate AI risk.
For more information, here is the whitepaper, and here is the dashboard.
5
