p(doom) is about doom. For AI, I think this can mean a few things:

• Literal human extinction

• Humans lose power over their future but are still alive (and potentially even have nice lives), either via stable totalitarianism or gradual disempowerment or other means

The second bucket is pretty big

What do the superforecasters say? Well, the most comprehensive effort to ascertain and influence superforecaster opinions on AI risk was the Forecasting Research Institute’s Roots of Disagreement Study.[2] In this study, they found that nearly all of the superforecasters fell into the “AI skeptic” category, with an average P(doom) of just 0.12%. If you’re tempted to say that their number is only so low because they’re ignorant or haven’t taken the time to fully understand the arguments for AI risk, then you’d be wrong; the 0.12% figure was obtained after having months of discussions with AI safety advocates, who presented their best arguments for believing in AI x-risks.

I see this a bunch but I think this study is routinely misinterpreted. I have some knowledge from having participated in it.

The question being posed to forecasters was about literal human extinction, which is pretty different from how I see p(doom) be interpreted. A lot of the "AI skeptics" were very sympathetic to AI being the biggest deal, but just didn't see literal extinction as that likely. I also have moderate p(doom) (20%-30%) while thinking literal extinction is much lower than that (<5%).

Also the study ran 2023 April 1 to May 31, which was just right after the release of GPT-4. Since then there's been so much more development. My guess is if you polled the "AI skeptics" now, the p(doom) will have gone up.

I just saw that Season 3 Episode 9 of Leverage: Redemption ("The Poltergeist Job") that came out on 2025 May 29 has an unfortunately very unflattering portrayal of "effective altruism".

Matt claims he's all about effective altruism. That it's actually helpful for Futurilogic to rake in billions so that there's more money to give back to the world. They're about to launch Galactica. That's free global Internet.

[...] But about 50% of the investments in Galactica are from anonymous crypto, so we all know what that means.

The main antagonist and CEO of Futurilogic, Matt, uses EA to justify horrific actions, including allowing firefighters to be injured when his company's algorithm throttles cell service during emergencies. He also literally murders people while claiming it's for the greater good. And if that's not enough, he's also laundering money for North Korea through crypto investments!

Why would he do this? He explicitly invokes utilitarian reasoning ("Trolley Theory 101") to dismiss harm caused:

When I started this company, I started it on the idea that if we could make enough money, we could make the entire world a better place, guys. All of it. Sometimes, in order to make something like that happen, huge sacrifices are required. Sacrifices like Josh. Or sacrifices like the firefighters. But that's Trolley Theory 101, guys. Yeah. I don't have any regrets. Not one.

And when wielding an axe to kill someone, Matt says: "This is altruism, Skylar! Whatever I need to do to save the world."

But what's his cause area? Something about ending "global hunger and homelessness" through free internet access. Matt never articulates any real theory of change beyond "make money (and do crimes) → launch free internet → somehow save world."

And of course the show depicts the EA tech executives at Futurilogic as being in a "polycule" with a "hive mind" mentality.

Bummer.

AGI by 2028 is more likely than not

I think it's 20% likely based on the model I made.

I don't think this is as clear of a dichotomy as people think it is. A lot of global catastrophic risk doesn't come from literal extinction because human extinction is very hard. A lot of mundane work on GCR policy involves a wide variety of threat models that are not just extinction.

Here's my summary of the recommendations:

• National security testing
  • Develop robust government capabilities to evaluate AI models (foreign and domestic) for security risks
  • Once ASL-3 is reached, government should mandate pre-deployment testing
  • Preserve the AI Safety Institute in the Department of Commerce to advance third-party testing
  • Direct NIST to develop comprehensive national security evaluations in partnership with frontier AI developers
  • Build classified and unclassified computing infrastructure for testing powerful AI systems
  • Assemble interdisciplinary teams with both technical AI and national security expertise
     
• Export Control Enhancement
  • Tighten semiconductor export restrictions to prevent adversaries from accessing critical AI infrastructure
  • Control H20 chips
  • Require government-to-government agreements for countries hosting large chip deployments
    • As a prerequisite for hosting data centers with more than 50,000 chips from U.S. companies, the U.S. should mandate that countries at high-risk for chip smuggling comply with a government-to-government agreement that 1) requires them to align their export control systems with the U.S., 2) takes security measures to address chip smuggling to China, and 3) stops their companies from working with the Chinese military. The “Diffusion Rule” already contains the possibility for such agreements, laying a foundation for further policy development.
  • Review and reduce the 1,700 H100 no-license required threshold for Tier 2 countries
    • Currently, the Diffusion Rule allows advanced chip orders from Tier 2 countries for less than 1,700 H100s —an approximately $40 million order—to proceed without review. These orders do not count against the Rule’s caps, regardless of the purchaser. While these thresholds address legitimate commercial purposes, we believe that they also pose smuggling risks. We recommend that the Administration consider reducing the number of H100s that Tier 2 countries can purchase without review to further mitigate smuggling risks.
  • Increase funding for Bureau of Industry and Security (BIS) for export enforcement
     
• Lab Security Improvements
  • Establish classified and unclassified communication channels between AI labs and intelligence agencies for threat intelligence sharing, similar to Information Sharing and Analysis Centers used in critical infrastructure sectors
  • Create systematic collaboration between frontier AI companies and intelligence agencies, including Five Eyes partners
  • Elevate collection and analysis of adversarial AI development to a top intelligence priority, as to provide strategic warning and support export controls
  • Expedite security clearances for AI industry professionals
  • Direct NIST to develop next-generation security standards for AI training/inference clusters
  • Develop confidential computing technologies that protect model weights even during processing
  • Develop meaningful incentives for implementing enhanced security measures via procurement requirements for systems supporting federal government deployments.
  • Direct DOE/DNI to conduct a study on advanced security requirements that may become appropriate to ensure sufficient control over and security of highly agentic models

• Energy Infrastructure Scaling
  • Set an ambitious national target: build 50 additional gigawatts of power dedicated to AI by 2027
  • Streamline permitting processes for energy projects by accelerating reviews and enforcing timelines
  • Expedite transmission line approvals to connect new energy sources to data centers
  • Work with state/local governments to reduce permitting burdens
  • Leverage federal real estate for co-locating power generation and next-gen data centers

• Government AI Adoption
  • across the whole of government, the Administration should systematically identify every instance where federal employees process text, images, audio, or video data, and augment these workflows with appropriate AI systems.
  • Task OMB to address resource constraints and procurement limitations for AI adoption
  • Eliminate regulatory and procedural barriers to rapid AI deployment across agencies
  • Direct DoD and Intelligence Community to accelerate AI research, development and procurement
  • Target largest civilian programs for AI implementation (IRS tax processing, VA healthcare delivery, etc.)

• Economic Impact Monitoring
  • Enhance data collection mechanisms to track AI adoption patterns and economic implications
  • The Census Bureau’s American Time Use Survey should incorporate specific questions about AI usage, distinguishing between personal and professional applications while gathering detailed information about task types and systems employed.
  • Update Census Bureau surveys to gather detailed information on AI usage and impacts
  • Collect more granular data on tasks performed by workers to create a baseline for monitoring changes
  • Track the relationship between AI computation investments and economic performance
  • Examine how AI adoption might reshape the tax base and cause structural economic shifts

If you've liked my writing in the past, I wanted to share that I've started a Substack: https://peterwildeford.substack.com/

Ever wanted a top forecaster to help you navigate the news? Want to know the latest in AI? I'm doing all that in my Substack -- forecast-driven analysis about AI, national security, innovation, and emerging technology!

Yeah I think so, though there still is a lot of disagreement about crucial considerations. I the OP advice list is about as close as it’s going to get.

I'm very uncertain about whether AI really is >10x neglected animals and I cannot emphasize enough that reasonable and very well-informed people can disagree on this issue and I could definitely imagine changing my mind on this over the next year. This is why I framed my comment the way I did hopefully making it clear that donating to neglected animal work is very much an answer I endorse.

I also agree it's very hard to know whether AI organizations will have an overall positive or negative (or neutral) impact. I think there's higher-level strategic issues that make the picture very difficult to ascertain even with a lot of relevant information (imo Michael Dickens does a good job of overviewing this even if I have a lot of disagreements). Also the private information asymmetry looms large here.

I also agree that "work that aims to get AI companies to commit towards not committing animal mistreatment" is an interesting and incredibly underexplored area. I think this is likely worth funding if you're knowledgable about the space (I'm not) and know of good opportunities (I currently don't).

I do think risk aversion is underrated as a reasonable donor attitude and does make the case for focusing on neglected animals stronger.

Since it looks like you're looking for an opinion, here's mine:

To start, while I deeply respect GiveWell's work, in my personal opinion I still find it hard to believe that any GiveWell top charity is worth donating to if you're planning to do the typical EA project of maximizing the value of your donations in a scope sensitive and impartial way. ...Additionally, I don't think other x-risks matter nearly as much as AI risk work (though admittedly a lot of biorisk stuff is now focused on AI-bio intersections).

Instead, I think the main difficult judgement call in EA cause prioritization right now is "neglected animals" (eg invertebrates, wild animals) versus AI risk reduction.

AFAICT this also seems to be somewhat close to the overall view of the EA Forum as well as you can see in some of the debate weeks (animals smashed humans) and the Donation Election (where neglected animal orgs were all in the top, followed by PauseAI).

This comparison is made especially difficult because OP funds a lot of AI but not any of the neglected animal stuff, which subjects the AI work to significantly more diminished marginal returns.

To be clear, AI orgs still do need money. I think there's a vibe that all the AI organizations that can be funded by OpenPhil are fully funded and thus AI donations are not attractive to individual EA forum donors. This is not true. I agree that their highest priority parts are fully funded and thus the marginal cost-effectiveness of donations is reduced. But this marginal cost-effectiveness is not eliminated, and it still can be high. I think there are quite a few AI orgs that are still primarily limited by money and would do great things with more funding. Additionally it's not healthy for these orgs to be so heavily reliant on OpenPhil support.

So my overall guess is if you think AI is only 10x or less important in the abstract than work on neglected animals, you should donate to the neglected animals due to this diminishing marginal returns issue.

I currently lean a bit towards AI is >10x neglected animals and therefore I want to donate to AI stuff, but I really don't think this is settled, it needs more research, and it's very reasonable to believe the other way.

~

Ok so where to donate? I don't have a good systematic take in either the animal space or the AI space unfortunately, but here's a shot:

For starters, in the AI space, a big issue for individual donors is that unfortunately it's very hard to properly evaluate AI organizations without a large stack of private information that is hard to come by. This private info has greatly changed my view of what organizations are good in the AI space. On the other hand you can basically evaluate animal orgs well enough with only public info, and the private info only improves the eval a little bit.

Moreover, in the neglected animal space, I do basically trust the EA Animal Welfare Fund to allocate money well and think it could be hard for an individual to outperform that. Shrimp Welfare Project also looks compelling.

I think the LTFF is worth donating to but to be clear I don't think the LTFF actually does all-considered work on the topic - they seem to have an important segment of expertise that seems neglected outside the LTFF, but they definitely don't have the expertise to cover and evaluate everything. I do think the LTFF would be a worthy donation choice.

If I were making a recommendation I would concur with the recommend the three AI orgs in OpenPhil's list: Horizon, ARI, and CLTR -- they are all being recommended by individual OpenPhil staff for good reason.

There are several other orgs I think are worth considering as well and you may want to think about options that are only available to you as an individual, such as political donations. Or think about ways where OpenPhil may not be able to do as well in the AI space, like PauseAI or digital sentience work, both of which still look neglected.

~

A few caveats/exceptions to my above comment:

• I'm very uncertain about whether AI really is >10x neglected animals and I cannot emphasize enough that reasonable and very well-informed people can disagree on this issue and I could definitely imagine changing my mind on this over the next year.

• I'm not shilling for my own orgs in this comment to keep it less biased, but those are also options.

• I don't mean to be mean to GiveWell. Of course donating to GiveWell is very good and still better than 99.99% of charitable giving!

• Another area I don't consider but probably should is organizations like Giving What We Can that work somewhat outside these cause areas but may have sufficient multipliers that it still is very cost-effective. I think meta-work on top of global health and development work (such as improving its effectiveness or getting more people to like it / do it better) can often lead to larger multipliers since there's magnitudes more underlying money in that area + interest in the first place.

• I don't appropriately focus on digital sentience, which OpenPhil is also not doing and could also use some help. I think this could be fairly neglected. Work that aims to get AI companies to commit towards not committing animal mistreatment is also an interesting and incredibly underexplored area that I don't know much about.

• There's a sizable amount of meta-strategic disagreement / uncertainty within the AI space that I gloss over here (imo Michael Dickens does a good job of overviewing this even if I have a lot of disagreements with his conclusions).

• I do think risk aversion is underrated as a reasonable donor attitude that can vary between donors and does make the case for focusing on neglected animals stronger. I don't think there's an accurate and objective answer about how risk averse you ought to be.