Preparing Effective Altruism for an AI-Transformed World

I think you make a really important point! You/anyone else interested in this may be interested in talking to @Constance Li and her work with @AI for Animals (Website)

Point 1: Broad agreement with a version of the original post's argument  

Thanks for this. I think I agree with you that people in the global health and animal spaces should, at the margin, think more about the possibility of Transformative AI (TAI), and short-timeline TAI. 

For animal-focussed people, maybe there’s an argument that because the default path of a non-TAI future is likely so bad for animals (eg persuading people to stop eating animals is really hard, persuading people to intervene to help wild animals is really hard, etc), that we might, actually, want to heavily “bet” on futures *with* TAI, because it’s only those futures which hold out the prospect of a big reduction in animal suffering. So we should optimise our actions for worlds where TAI happens, and try to maximise the chances that these futures go very well for non-human animals. 

I think this is likely less true for global health and wellbeing, where plausibly the global trends look a lot better.

Point 2: Some reasons to be sceptical about claims of short-timeline Transformative AI 

Having said that, there’s something about the apparent certainty that “TAI is nigh” in the original post, which prompted me to want to scribble down some push-back-y thoughts. Below are some plausible-sounding-to-me reasons to be sceptical about high-certainty claims that TAI is close. I don’t pretend that these lines of thought in-and-of-themselves demolish the case for short-timeline TAI, but I do think that they are worthy of consideration and discussion, and I’d be curious to hear what others make of them:

• The prediction of short-timeline TAI is based on speculation about the future. Humans very often get this type of speculation wrong.
• Global capital markets aren’t predicting short-timeline TAI. A lot of very bright people, who are highly incentivised to make accurate predictions about the future shape of the economy, are not betting that TAI is imminent.
• Indeed, most people in the world don’t seem to think that TAI is imminent. This includes tonnes of *really* smart people, who have access to a lot of information.
• There’s a rich history of very clever people making bold predictions about the future, based on reasonable-sounding assumptions and plausible chains of reasoning, which then don’t come true - e.g. Paul Ehrlich’s Population Bomb.
• Sometimes, even the transhumanist community - where notions of AGI, AI catastrophe risk, etc, started out - get excited about a certain technological risk/trend, but then it turns out not to be such a big deal - e.g. nanotech, “grey goo”, etc in the ‘80s and ‘90s.
• In the past, many radical predictions about the future, based on speculation and abstract chains of reasoning, have turned out to be wrong.
• Perhaps there’s a community effect whereby we all hype ourselves up about TAI and short timelines. It’s exciting, scary and adrenaline-inducing to think that we might be about to live through ‘the end of times’.
• Perhaps the meme of “TAI is just around the corner/it might kill us all” has a quality which is psychologically captivating, particularly for a certain type of mind (eg people who are into computer science, etc); perhaps this biases us. The human mind seems to be really drawn to “the end is nigh” type thinking.
• Perhaps questioning the assumption of short-timeline TAI has become low-status within EA, and potentially risky in terms of reputation, funding, etc, so people are disincentivised to push back on it. 

To restate: I don’t think any of these points torpedo the case for thinking that TAI is either inevitable, and/or imminent. I just think they are valid considerations when thinking about this topic, and are worthy of consideration/discussion, as we try to decide how to act in the world. 

My current view of that is best summarized by: "Drop the 'AI'. Just policy. It's cleaner"

One GHW example: The impact of AI tutoring on educational interventions (via Arjun Panickssery on LessWrong). 

There have been at least 2 studies/impact evaluations of AI tutoring in African countries finding extraordinarily large effects:

Summer 2024 — 15–16-year olds in Nigeria
They had 800 students total. The treatment group studied with GPT-based Microsoft Copilot twice weekly for six weeks, studying English. They were just provided an initial prompt to start chatting—teachers had a minimal “orchestra conductor” role—but they achieved “the equivalent of two years of typical learning in just six weeks.”

February–August 2023 — 8–14-year-olds in Ghana
An educational network called Rising Academies tested their WhatsApp-based AI math tutor called Rori with 637 students in Ghana. Students in the treatment group received AI tutors during study hall. After eight months, 25% of the subjects attrited from inconsistent school attendance. Of the remainder, the treatment group increased their scores on a 35-question assessment by 5.13 points versus 2.12 points for the control group. This difference was “approximately equivalent to an extra year of learning” for the treatment group.
 

Should this significantly change how excited EAs are about educational interventions? I don't know, but I've also not seen a discussion of this on the forum (this post about MOOC & AI tutors that received ~zero engagement).

+1. I appreciated @RobertM’s articulation of this problem for animal welfare in particular:

I think the interventions for ensuring that animal welfare is good after we hit transformative AI probably look very different from interventions in the pretty small slice of worlds where the world looks very boring in a few decades.

…

If we achieve transformative AI and then don’t all die (because we solved alignment), then I don’t think the world will continue to have an “agricultural industry” in any meaningful sense (or, really, any other traditional industry; strong nanotech seems like it ought to let you solve for nearly everything else). Even if the economics and sociology work out such that some people will want to continue farming real animals instead of enjoying the much cheaper cultured meat of vastly superior quality, there will be approximately nobody interested in ensuring those animals are suffering, and the cost for ensuring that they don’t suffer will be trivial.

[...] if you think it’s at all plausible that we achieve TAI in a way that locks in reflectively-unendorsed values which lead to huge quantities of animal suffering, that seems like it ought to dominate effectively all other considerations in terms of interventions w.r.t. future animal welfare.

I’ve actually tried asking/questioning a few animal welfare folks for their takes here, but I’ve yet to hear back anything that sounded compelling (to me). (If anyone reading this has an argument for why ‘standard’ animal welfare interventions are robust to the above, then I’d love to hear it!)

A lot of EAs do think AI safety could become ridiculously important (i.e. some probability mass of very short timelines) but are not in the position to do anything, which is why they focus on more tractable areas (i.e. global health, animal welfare, EA building) under the assumption of longer AI timelines. Especially because there's a lot of uncertainty about when AGI would come.

My internal view is 25% of TAI by 2040 and 50% of TAI by 2060, where I define TAI as an AI with the ability to autonomously perform AI research. They may have shifted in light of DeepSeek but what am I supposed to do? I'm just a freshman college student at a non-prestigious university. Am I supposed to drop all commitments I have, speed-run my degree, get myself to work in a highly competitive AI lab which would probably require a Ph. D., work on technical alignment hoping to get a breakthrough? If TAI comes within 5 years, it would be the right move, but if I'm wrong then I would end up with very shallow skills without much experience.

We have the following Pascal matrix (drafted my GPT):

I know the decision is not binary, but I am definitely willing to forfeit 25% of my impact by betting on the AGI comes late scenario. I do think non-AI cause areas should use AI projection in their deliberation and ToC but I think it is silly to cut out everything that happens after 2040 with respect to the cause area. 

However, I do think EAs should have a contingency plan where they should speedrun to AI safety if and only if (one of multiple conditions occur; i.e. even conservative superforecastors project AGI before 2040, or a national emergency is declared). And we can probably hedge against the AGI comes soon scenario by buying long-term NVIDIA call options.

If I wasn’t working on AI Safety I’d work on near term (< 5 years) animal welfare interventions.

Strong upvote! I want to say some stuff particularly within the context of global development:

The intersection of AI and global development seems surprisingly unsaturated within EA, or to be more specific, I think a surprisingly few number of EAs think about the following questions:

i) How to leverage AI for development (e.g. AI tools for education, healthcare)  
ii) What interventions and strategies should be prioritized within global health and development in the light of AI developments? (basically the question you ask)

There seems to be a lot of people thinking about the first question outside of EA, so maybe that explains this dynamic, but I have the "hunch" that the primary reason why people don't focus on the first question too much is people deferring too much and selection effects, rather lack of any high-impact interventions. If you care about TAI, you are very likely to work on AI alignment & governance, if you don't want to work on TAI-related things (due to risk-aversion or any other argument/value), you just don't update that much based on AI developments and forecasts. This may also have to do with EA's ambiguity-averse/risk-averse attitude towards GHD characterized by exploiting evidence-based, interventions rather than exploring new highly promising interventions. I think if a student/professional were to come to an EA community-builder and asked "How can I pursue a high-impact career in/upskill in global health R&D or AI-for-development", number of community-builders that can give a sufficiently helpful answer is likely very few to none, I also likely wouldn't be able to give a good answer and point to communities/resources outside of the EA community. 

(Maybe EAs in London or SF people discuss these, but I don't see any discussion of it online, neither do I see any spaces where people who could be discussing these can network/discuss together. If there is anyone who'd like to help create or run an online or in-person AI-for-development or global health R&D fellowship, feel free to shoot a message) 


 

Many people believe that AI will be transformative, but choose not to work on it due to factors such a (perceived) lack of personal fit or opportunity, personal circumstances, or other practical considerations.

There may be various other reasons why people choose to work on other areas, despite believing transformative AI is very likely, e.g. decision-theoretic or normative/meta-normative uncertainty.

I gave this a strong upvote because regardless of whether or not you agree with these timelines or Tobias' conclusion, this is a discussion that the community needs to be having. As in, it's hard to argue that the possibility of this is remote enough these days that it makes sense to ignore it.

I would love to see someone running a course focusing on this (something broader than the AI Safety Fundamentals course). Obviously this is speculative, but I wouldn't be surprised if the EA Infrastructure Fund were interested in funding a high-quality proposal to create such a course.

Hi Tobias.

If we accept that AI is likely to reshape the world over the next 10–15 years, this realisation will have major implications for all cause areas.

I think donating to the Shrimp Welfare Project (SWP) would still have super high cost-effectiveness even if the world was certain to end in 10 years. I estimate it has been 64.3 k times as cost-effective as GiveWell’s top charities (ignoring their effects on animals) for 10 years of acceleration of the adoption of electrical stunning, as used by Open Philanthropy (OP). If the acceleration followed a normal distribution, SWP's cost-effectiveness would only become 50 % as high if the world was certain to end in 10 years. I think this would still be orders of magnitude more cost-effective than the best interventions in global health and development and AI safety.

There is also the question of whether the world will actually be radically reshaped. I am happy to bet 10 k$ against short timelines for that.

A key implication here is that we need models of how AI will transform the world with many qualitative and quantitative details. Individual EAs working in global health, for example, cannot be expected to broadly predict how the world will change.

My view, having thought about this a fair bit, is that there is an extremely broad range of outcomes ranging from human extinction, to various dystopias, to utopia or "utopia". But there are probably a lot of effects that are relatively predictable, especially in the near term.

Of course, EAs in field X can think about how AI affects X. But this should work better after learning about whatever broad changes superforecasters (or whoever) can predict.

Instead of framing priorities this way, I believe it would be valuable for more people to adopt a mindset that assumes transformative AI is likely coming and asks: What should we work on in light of that?

For climate change, I think it means focusing on the catastrophes that could plausibly happen in the next couple decades, such as coincident extreme weather on multiple continents or the collapse of the sub polar gyre. So adaptation becomes relatively more important than emissions reductions. Since AI probably makes nuclear conflict and engineered pandemic more likely, the importance of these fields may still be similar, but you would likely move away from long payoff actions like field building and maybe some things that are likely to take a long time, such as major arsenal reductions or wide adoption of germicidal UV or enhanced air filtration. Instead, one might focus on trying to reduce the chance of nuclear war, especially given AI-enabled systems or AI-induced tensions, or increasing resilience to nuclear war. On the pandemic side, of course reducing the risk that AI enables pandemics, but also near-term actions that could have a significant impact like early warning for pandemics or rapid scale up of resilience post outbreak.

I very strongly agree with integrating the likelihood of TAI with other cause areas. 

Animal welfare being my primary cause area, I have found it somewhat odd that this is a separate cause area in itself to the other anthropocentric cause areas (though it makes sense for it to be for emphasis' sake). In reality I think there's inevitably an intersection between cause areas like AI safety x Animals and Longtermism x Animals; it's just that AI safety and longtermism currently hold a very anthropocentric focus, but in principle these could be applied to animal welfare.

I get that AI could reshape everything, but does that mean we should deprioritize issues like factory farming, poverty, or malaria problems causing real suffering right now? AI isn’t inherently good or bad; it depends on how humans use it. If anything, shouldn’t we be asking how AI can support existing EA causes rather than assuming everything else becomes irrelevant? Feels like we might be overcorrecting here.

I’m not sure how plausible and feasible it is, but I believe using AI to automate data collection, integration, and analysis could potentially streamline processes like diagnosing problems, testing hypotheses, and prioritizing interventions in GHW, at least in some domains.