I got interested in effective altruism back before it was called effective altruism, back before Giving What We Can had a website. Later on, I got involved in my university EA group and helped run it for a few years.
I joined the Effective Altruism Forum to try to figure out where effective altruism could fit into my life these days and what it means to me. You can read my latest thoughts on effective altruism here.
I write on Substack, and used to write on Medium.
Pronouns: she/her or they/them.
I first learned about these predictions from a post on the philosopher David Thorstad's fantastic blog Reflective Altruism. Thorstad, in turn, cited this post on the EA Forum. Credit where credit is due!
There is some background that will unfortunately take a while to explain, but it seems worthwhile.
Eliezer Yudkowsky has a very odd approach where he completely changes his mind on big topics, but never really explains why, if he even acknowledges that it happened. His fear that nanotech would imminently end the world is one. Unless I'm missing something, he never explained why he changed his mind on this, besides a few short oblique comments.
You'd think it would be a big deal. He used to say that we need to rush to build AGI as fast as possible to stop the world from imminent destruction from nanotech. Now he says countries should agree to stop working on AGI and blow up the datacentres of any countries that don't comply. That's a huge shift! What happened? There's no real explanation that I can find...
Another one is deep learning or deep neural networks. In 2008, Yudkowsky wrote a post that was critical of neural networks. Even into the mid-2010s, several years into the deep learning revolution that kicked off in 2012, MIRI was still primarily focused on hand-coded AI. I think it wasn't until after AlphaGo defeated Lee Sedol in 2016 that MIRI finally hired its first research intern to focus on machine learning. By no later than 2023, Yudkowsky was fully bought-in to deep learning. But what happened? When and why did he change his mind?
I've searched extensively and asked around. Nobody seems to have a clear answer on this. Even people from MIRI or Lightcone Infrastructure (the organization that runs LessWrong) don't seem to know. A member of the LessWrong community — presumably a friendly audience — wrote a critical review of the book Yudkowsky co-authored, focusing mainly on this point. Yudkowsky's talking points about AI safety are essentially the same as they were in 2008 when he was skeptical of neural networks. They have been cosmetically updated for the deep learning era. But he assumed in 2008 (and seemingly for some time after) that AGI would be hand-coded. Why should ideas about hand-coded AI still apply to deep learning? If they still apply, why do they? What's the reasoning? What's the argument? Yudkowsky never says.
The unfortunate situation is that LessWrong is essentially an Eliezer Yudkowsky fan club founded by Eliezer Yudkowsky. Yudkowsky's blog posts called The Sequences are a revered text. Even his Harry Potter fanfic is a revered text. Yudkowsky claims he is the smartest person in the world. Many of his fans seem to agree.
In a normal context, people would evenhandedly evaluate Yudkowsky's views and track record on topics like AI and nanotech. But he has many followers who see him as some kind of one-of-a-kind world-historical genius, some kind of nearly infallible mystical guru figure. He's treated with reverence and loyalty rather than critical thinking and accountability. Not by everyone, but by a lot of people.
I don't buy EA's self-congratulatory self-image, which you described:
There are many reasons I don't buy this self-image, some of which I've detailed here and here. You can look at the Yudkowsky fan club as one example (among many) of why that self-image doesn't add up.
With the dot-com bubble, there was a lag of about 10 years between when the bubble popped and companies like Amazon recovered to their bubble-era stock prices. How long do you think the lag with large language models will be? 5 years, 10 years, 20 years, 30 years...?
Personally, I just don't see LLMs ever becoming useful enough to justify $5 trillion in market cap or whatever it is. They aren't that useful now and they aren't improving that much. As a technology, they seem like a dead end.
The term AI is incredibly broad. It was coined in 1955 and has been used to describe many different kinds of science and technology since then. If science and technology continue to make progress, then it seems likely that eventually some technology that can rightly be called AI will have a huge, durable economic impact — whether that's in 20 years or 120 years or 200 years. But LLMs most likely won't ever have that kind of impact, even 20 years from now.
Wanna bet? :)
I have an open offer to bet anyone who'd like that AI is in a bubble and that the bubble will pop within the next ~7 years.
The timeframe is so long because bubbles are impossible to time precisely. Hopefully the true outcome will not take so long to become clear. But you never know.
The purpose of the bet is to do just what Siobhan said in this post:
Plus, betting is fun!
Good post. It's worth mentioning that this already happened. This is what the Singularity Institute, now called the Machine Intelligence Research Institute (MIRI), wrote in 2000. MIRI predicted that transhuman AI — what we'd now call artificial general intelligence — would be invented no later than 2020, and probably much earlier:
In 1999, Eliezer Yudkowsky, MIRI's founder, wrote that he believed dangerous nanotechnology would wipe out humanity unless we (or, specifically he) invented friendly AI first. (Now he says don't just slow down, stop. Back then he said we had to go fast.) Nowadays, we'd call friendly AI something like aligned AGI or safe AGI. Yudkowsky's name for his friendly AI project was "Ellison".
He wrote:
So, what happens when people get the timeline wrong by at least 6 years? You've seen one example of what has happened! Look at MIRI and at Eliezer Yudkowsky, and at the LessWrong community.
Another example I personally found interesting to look at is the eschatology of Jehovah's Witnesses. In the 1960s, the Jehovah's Witnesses leadership embraced an end of the world prophecy with a specific year attached: 1975.
This prophecy was widely taught and believed among Jehovah's Witnesses. People took this deadly seriously and did things like sell their house and devote themselves full-time to evangelism.
When 1976 rolled around and the world was not over, the church leadership backpedaled and did revisionist history on what it had actually said. Recruitment suffered greatly for at least a few years, although the church did not collapse and obviously still exists. One leader even said the prophecy's falsification was a good thing, since it separated the true faithful wheat from the wavering chaff.
Would it be so surprising if the most devoted Singularity believers kept kicking the can down the road over and over forever until they eventually died of old age 50 or 60 years from now? Not to me, at least. That seems perfectly human.
On the other hand, many people whose belief is not as strong will probably rethink things when the AI bubble pops.
In the late 2000s and early 2010s, GiveWell would look at meta-analyses of randomized controlled trials and get lots of expert input on whether they were interpreting them correctly. This was the sort of approach that was supposed to typify effective altruism.
Fast forward to the mid-2020s and now subjective intuition plays an unreasonably large role in how effective altruism attempts to answer empirical, quantitative questions of a scientific or technical nature. What happened to effective altruism?
Worse, it's become a recurring problem that subjective intuitions are deceptively dressed up as objective empirical evidence. It's bad enough to leave behind more rigorous, scientific approaches in favour of subjective intuition. It's even worse to mislead people — either willingly or through carelessness — into thinking you're reporting objective, measured data when you're really just reporting your subjective guesses.
This shift away from science toward intuition feels even more troubling when you connect it to the hostility expressed by some members of the EA community toward academia, journalism, and liberal democracy. There is a sort of dark, conspiratorial, fringe worldview you get when you combine these troubling tendencies. When considering this, it should be remembered that there have been violent killings and threats of killings at the peripheries of the EA community. This does not normally happen with mainstream social movements of this size. What is fuelling these extremist acts?
More and more, I appreciate just how important are the intellectual cornerstones of modern, Enlightenment society like science and liberalism, and the institutions that support them, like academia and journalism. Extremism and darkness are only a few short steps away from the light. The barrier between them is as thin as air.
May those who carry the light win, and may those lost in the darkness find their way back.
For Anthropic to become the world's most profitable company, LLMs would need to have a much greater impact on productivity than they do currently. Previously, there was a lot of excitement that LLMs would become much more generally capable through scaling. Now that excitement seems to have greatly diminished.
In 2024, Ilya Sutskever, a prominent AI researcher and a co-founder of OpenAI, declared that the age of scaling is over. He said that AI must return to an age of research, rather than scaling. In 2025, Sutskever reiterated this point of view. I haven't seen other prominent people at AI companies state it so starkly. But I have noticed a dissipation of the excitement around scaling. Few people (if any) seem to talk about scaling with the optimism they used to.
This leaves the question: if not scaling, then what would make LLMs much more generally capable within the next 5 to 10 years?
Two possible answers are:
Why should we believe recursive self-improvement will happen within the next 5 to 10 years? LLMs can't fully automate ostensibly much simpler jobs like translation or customer support. Why would AI research be so easy to automate? Why would that particular job be automated when no other job has been?
Many forms of automated machine learning or AutoML have been pursued for many years. The results have been quite modest compared to what people imagine when they imagine recursive self-improvement. Why would AutoML suddenly have a breakout success now, when it hasn't for so many years?
LLMs have become important in coding and certain forms of math research, two domains where they're extremely naturally well-suited to succeed. We may see LLMs open up a few more niches in the future — or maybe not. But this is distinct from the vision that LLMs are a form of general-purpose, generally useful AI that will enhance the productivity of everything everywhere all at once. That vision shows no signs of coming to fruition. Instead we see what we've seen with previous advances over the last 14 years of deep learning: AI systems that are really good at a few specific things.
Okay, but what does that have to do with EA? What does that have to do with my original comment? What point were you trying to make? I don't see the logical connection.
You were the one who brought up the dollar amounts! Why did you do that?
I'm not sure why you brought up this point about political influence in the first place, but incidentally, it helps my point.
Culture, leadership, and ethics can have huge downstream consequences if a social movement gains political influence. There are two important considerations there:
This leads us to ask: is EA's culture healthy? Do EA orgs and EA leaders have good responses to problems within the movement? Are EA orgs, EA leaders, and prominent individuals in EA ethically upstanding? If they do illegal or unethical things, is there accountability and consequences from the EA movement?
There are many serious concerns along these lines, such as:
In light of things like these, reasonable people ought to have serious doubts about whether the culture and leadership of EA can be trusted.
Should you trust the Centre for Effective Altruism, which was once under the control of a cult?
Should you trust the EA Forum, which allows the promotion of scientific racism?
Should you trust Stop AI, which came perilously close to becoming effectively a domestic terrorist organization?
Should you trust Manifund, which is engaged in brazen financial corruption?
Should you trust 80,000 Hours, which blatantly misleads its viewers on YouTube?
No, probably not!
It's nothing personal. My loyalty is to the world and to doing the right thing. Regardless of whether doing the right thing benefits EAs — emotionally, financially, or otherwise. Nothing against your friends, but the purpose of EA is to benefit the global poor and factory farmed animals, not your friends.
This is why entangling your personal identity and self-esteem with effective altruism can be so dangerous. It can make you unwilling or literally emotionally unable to take problems with EA seriously and work to correct them.
I don't follow your logic.
What does the scale of USAID cuts have to do with anything? The USAID cuts are massive and terrible. They are massive and terrible compared to all private giving, including all effective altruism-related giving. The USAID cuts under the second Trump admin are well in excess of $11 billion a year, which is apparently the total amount that was given privately in 2023. Figures from a few years ago via Our World in Data:
I don't know if EAGrantsDatabase.org is accurate, but it tracks less than $1.2 billion in grants per year. Grants for global health and development are said to be around $400 million to $500 million a year. According to Bloomberg, Dustin Moskovitz and Cari Tuna's net worth is around $13 billion total, which is less than one year of USAID cuts.
According to two different methodologies, one used in 2024 (see page 6), and one used in 2025 (see page 7), GiveWell estimates that it directs or influences at most around $600 million a year. However, it changed its methodology in 2025 to make it more conservative.
So, what was the point of bringing up USAID? Effective altruism can't compete with the scale of USAID, either. If you were trying to say this is why people should get involved in politics rather than private philanthropy, I would understand your point. Why is this a justification for EA? What does it have to do with anything?
Yep, it would be a lot of revenue, very fast!
In the older thread I linked to, a few people's rationale for forecasting that much revenue that fast seemed to be: just look at the graph! But you can look at literally thousands of graphs about companies' financial performance and note that their periods of fastest growth do not sustain indefinitely...
If there is a good rationale for thinking Anthropic will do $514 billion in 2027, whatever it is, it's not "just look at the graph".
If you want to know Anthropic's annualized revenue, the simplest way is just to take its most recent quarter's revenue and multiply by four. Q3 revenue is not disclosed in the S-1, but Reuters reported it at $16.8 billion. Times four that's $67.2 billion.
Something that could be useful as a guidepost is a) if Anthropic gives its own revenue estimates for 2027 and 2028 (e.g., Reuters reported in August that that Anthropic estimated $190 billion to $200 billion for 2028) and b) when Wall Street inevitably publishes its estimates. Anthropic and Wall Street know how to look at graphs but will not forecast $514 billion in revenue in 2027.