I'm not sure how they decide on what salaries to pay themselves. But the reason they have the money to pay themselves those salaries in the first place is that MIRI's donors believe there's a significant chance of AI destroying the world within the next 5-20 years and that MIRI (especially Yudkowsky) is uniquely positioned to prevent this from happening.

People who have radical anti-institutionalist views often take reasonable criticisms of institutions and use them to argue for their preferred radical alternative. There are many reasonable criticisms of liberal democracy; these are eagerly seized on by Marxist-Leninists, anarchists, and right-wing authoritarians to insist that their preferred political system must be better. But of course this conclusion does not necessarily follow from those criticisms, even if the criticisms are sound. The task for the challenger is to support the claim that their preferred system is robustly superior, not simply that liberal democracy is flawed.

The same is true for radical anti-institutionalist views on institutional science (which the LessWrong community often espouses, or at least whenever it suits them). Pointing out legitimate failures in institutional science does not necessarily support the radical anti-institutionalists' conclusion that peer-reviewed journals, universities, and government science agencies should be abandoned in favour of blogs, forums, tweets, and self-published reports or pre-prints. On what basis can the anti-institutionalists claim that this is a robustly superior alternative and not a vastly inferior one?

To be clear, I interpret you as making a moderate anti-institutionalist argument, not a radical one. But the problem with the reasoning is the same in either case — which is why I'm using the radical arguments for illustration. The guardrails in academic publishing sometimes fail, as in the case of research misconduct or in well-intentioned, earnestly conducted research that doesn't replicate as you mentioned. But is this an argument for kicking down all guardrails? Shouldn't it be the opposite? Doesn't this just show us that deeply flawed research can slip under the radar? Shouldn't this underscore the importance of savvy experts doing close, critical readings of research to find flaws? Shouldn't the replication crisis remind of us of the importance of replication (which has always been a cornerstone of institutional science)? Why should the replication crisis be taken as license to give up on institutions and processes that attempt to enforce academic rigour, including replication?

In the case of both AI 2027 and the METR graph, half of the problem is the underlying substance — the methodology, the modelling choices, the data. The other half of the problem is the presentation. Both have been used to make bold, sweeping, confident claims. Academic journals referee both the substance and the presentation of submitted research; they push back on authors trying to use their data or modelling to make conclusions that are insufficiently supported.

In this vein, one of the strongest critiques of AI 2027 is that it is an exercise in judgmental forecasting, in which the authors make intuitive, subjective guesses about the future trajectory of AI research and technology development. There's nothing inherently wrong with a judgmental forecasting exercise, but I don't think the presentation of AI 2027 is clear enough that AI 2027 is nothing more than that. (80,000 Hours' video on AI 2027, which is 34 minutes long and was carefully written and produced at a cost of $160,000, doesn't even mention this.)

If AI 2027 had been submitted to a reputable peer-reviewed journal, besides hopefully catching the modelling errors, the reviewers probably would have insisted the authors make it clear from the outset what data the conclusions are based on (i.e. the authors' judgmental forecasts) and where that data came from. They would probably also have insisted the conclusions are appropriately moderated and caveated in light of that. But, overall, I think AI 2027 would probably just be unpublishable.

I don't really like accusations of motivated reasoning. The logic you presented cuts both ways.

MIRI's business model relies on the opposite narrative. MIRI pays Eliezer Yudkowsky $600,000 a year. It pays Nate Soares $235,000 a year. If they suddenly said that the risk of human extinction from AGI or superintelligence is extremely low, in all likelihood that money would dry up and Yudkowsky and Soares would be out of a job.

The financial basis for motivated reasoning is arguably even stronger in MIRI's case than in Mechanize's case. The kind of work MIRI is doing and the kind of experience Yudkowsky and Soares have isn't really transferable to anything else. This means they are dependent on people being scared of enough of AGI to give money to MIRI. On the other hand, the technical skills needed to work on trying to advance the capabilities of current deep learning and reinforcement learning systems are transferable to working on the safety of those same systems. If the Mechanize co-founders wanted to focus on safety rather than capabilities, they could.

I'm also guessing the Mechanize co-founders decided to start the company after forming their views on AI safety. They were publicly discussing these topics long before Mechanize was founded. (Conversely, Yudkowsky/MIRI's current core views on AI were formed roughly around 2005 and have not changed in light of new evidence, such as the technical and commercial success of AI systems based on deep learning and deep reinforcement learning.)

I would consider driving people to delusion and suicide, killing people for self-preservation and even Hitler the man himself to be at least a somewhat "alien" style of evil.

The Yudkowsky/Soares/MIRI argument about AI alignment is specifically that an AGI's goals and motivations are highly likely to be completely alien from human goals and motivations in a way that's highly existentially dangerous. If you're making an argument to the effect that 'humans can also be misaligned in a way that's extremely dangerous', I think, at that point, you should acknowledge you've moved on from the Yudkowsky/Soares/MIRI argument (and maybe decided to reject it). You're now making a quite distinct argument that needs to be evaluated independently. It may be worth asking what to do about the risk that powerful AI systems will have human-like goals and motivations that are dangerous in the same way that human goals and motivations can be dangerous. But that is a separate premise from what Yudkowsky and Soares are arguing.

Misinformation and clickbait are also common ways to get attention. I wouldn’t recommend those tactics, either.

The way that a lot of people get attention online is fundamentally destructive. It gets them clicks and ad revenue, but it doesn’t help cause positive change in the world.

I don’t think it makes sense to justify manipulative, dishonest, or deceptive tactics like ragebait on the basis that they are good at getting attention. This is taking a business model from social media, which in some cases is arguably like digital cigarettes, and inappropriately applying it to animal advocacy. If the goal is to get people to scroll a lot and show them a lot of ads, sure, copy the tactics used in social media. But that isn’t the goal here.

One form of ragebait is when you generate rage at a target other than yourself, but another form is when you bait people into getting angry at you (e.g. by expressing an insincere opinion) because that drives engagement, and engagement gets you paid. Making people angry at you is especially not applicable to animal advocacy.

I don't know much about this topic myself, but my understanding is that market efficiency is less about having the objectively correct view (or making the objectively right decision) and more about the difficulty of any individual investor making investments that systematically outperform the market. (An explainer page here helps clarify the concept). So, the concept, I think, is not that the market is always right, but when the market is wrong (e.g. that generative AI is a great investment), you're probably wrong too. Or, more precisely, that you're unlikely to be systematically right more often then the market is right, and systematically wrong less often than the market is wrong.

As I understand it, there are differing views among economists on how efficient the market really is. And there is the somewhat paradoxical fact that people disagreeing with the market is part of what makes it as efficient as it is in the first place. For instance, some people worry that the rise of passive investing (e.g. via Vanguard ETFs) will make the market less efficient, since more people are just deferring to the market to make all the calls, and not trying to make calls themselves. If nobody ever tried to beat the market, then the market would become completely inefficient.

There is an analogy here to forecasting, with regard to epistemic deference to other forecasters versus herding that throws out outlier data and makes the aggregate forecast less accurate. If all forecasters just circularly updated until all their individual views were the aggregate view, surely that would be a big mistake. Right?

Do you have a specific forecast for AGI, e.g. a median year or a certain probability within a certain timeframe?

If so, I'd be curious to know how important AI investment is to that forecast. How much would your forecast change if it turned out the AI industry is in a bubble and the bubble popped, and the valuations of AI-related companies dropped significantly? (Rather than trying to specifically operationalize "bubble", we could just defer the definition of bubble to credible journalists.)

There are a few different reasons you've cited for credence in near-term AGI — investment in AI companies, the beliefs of certain AI industry leaders (e.g. Sam Altman), the beliefs of certain AI researchers (e.g. Geoffrey Hinton), etc. — and I wonder how significant each of them is. I think each of these different considerations could be spun out into its own lengthy discussion.

I wrote a draft of a comment that addresses several different topics you raised, topic-by-topic, but it's far too long (2000 words) and I'll have to put in a lot of work if I want to revise it down to a normal comment length. There are multiple different rabbit holes to go down, like Sam Altman's history of lying (which is why the OpenAI Board fired him) or Geoffrey Hinton's belief that LLMs have near-human-level consciousness.

I feel like going deeper into each individual reason for credence in near-term AGI and figuring out how significant each one is for your overall forecast could be a really interesting discussion. The EA Forum has a little-used feature called Dialogues that could be well-suited for this.

Because timelines are so uncertain (3-15 years?)

In fact, they are much more uncertain than that. AI researchers and superforecasters tend to guess a range between around 20 and 90 years for the median year of AGI.

100 megaseconds (why that unit of measurement?) is 3 years and 2 months. 100 megaseconds from now is March 2029. This is way earlier than the median year of AGI guessed by AI researchers and superforecasters. It's even earlier than Metaculus, which is disproportionately used by people who strongly believe in near-term AGI. Metaculus currently says 2033.

(My personal forecast, for whatever it's worth, is a significantly less than 0.01%, or 1 in 10,000, chance of AGI by the end of 2035 and a ~95% chance that the AI industry is in a bubble that will pop sometime within the next ~5 years.)

[Adapted from this comment.]

Two pieces of evidence commonly cited for near-term AGI are AI 2027 and the METR time horizons graph. AI 2027 is open to multiple independent criticisms, one of which is its use of the METR time horizons graph to forecast near-term AGI or AI capabilities more generally. Using the METR graph to forecast near-term AGI or AI capabilities more generally is not supported by the data and methodology used to make the graph.

Two strong criticisms that apply specifically to the AI 2027 forecast are:

• It depends crucially on the subjective intuitions or guesses of the authors. If you don't personally share the authors' intuitions, or don't personally trust that the authors' intuitions are likely correct, then there is no particular reason to take AI 2027's conclusions seriously.
• Credible critics claim that the headline results of the AI 2027 timelines model are largely baked in by the authors' modelling decisions, irrespective of what data the model uses. That means, to a large extent, AI 2027's conclusions are not actually determined by the data they use. We already saw with the previous bullet point that the conclusions of AI 2027 are largely a restatement of the authors' personal and contestable beliefs. This is another way in which AI 2027's conclusions are, effectively, a restatement of the pre-existing beliefs or assumptions that the authors chose to embed in their timelines model.

AI 2027 is largely based on extrapolating the METR time horizons graph. The following criticisms of the METR time horizons graph therefore extend to AI 2027:

• Some of the serious problems and limitations of the METR time horizons graph are sometimes (but not always) clearly disclosed by METR employees. Note the wide difference between the caveated description of what the graph says and the interpretation of the graph as a strong indicator of rapid, exponential improvement in general AI capabilities.
• Gary Marcus, a cognitive scientist and AI researcher, and Ernest Davis, a computer scientist and AAAI fellow, co-authored a blog post on the METR graph that looks at how the graph was made and concludes that “attempting to use the graph to make predictions about the capacities of future AI is misguided”.
• Nathan Witkin, a research writer at NYU Stern’s Tech and Society Lab, published a detailed breakdown of some of the problems with METR’s methodology. He concludes that it’s “impossible to draw meaningful conclusions from METR’s Long Tasks benchmark” and that the METR graph “contains far too many compounding errors to excuse”. Witkin calls out a specific tweet from METR, which presents the METR graph in the broad, uncaveated way that the AI 2027 authors interpret it. He calls the tweet “an uncontroversial example of misleading science communication”. 

Since AI 2027 leans so heavily on this interpretation of the METR graph to make its forecast, it is hard to see how AI 2027 could be credible if its interpretation of the METR graph is not credible. 

It's worth contrasting AI 2027 and similar forecasts of near-term AGI with expert opinion:

• 76% of AI experts think it is unlikely or very unlikely that existing approaches to AI, which includes LLMs, will scale to AGI. (See page 66 of the AAAI 2025 survey. See also the preceding two pages about open research challenges in AI — such as continual learning, long-term planning, generalization, and causal reasoning — none of which are about scaling more, or at least not uncontroversially so. If you want an example of a specific, prominent AI researcher who emphasizes the importance of fundamental AI research over scaling, Ilya Sutskever believes that further scaling will be inadequate to get to AGI.)
• Expert surveys about AGI timelines are not necessarily reliable, but the AI Impacts survey in late 2023 found that AI researchers’ median year for AGI is 20 to 90 years later than the AI 2027 scenario.

Two overall takeaways:

• There are good reasons to be highly skeptical of AI 2027 and the METR time horizons graph as evidence for near-term AGI or for a rapid, exponential increase in general AI capabilities.
• Peer review in academic research is designed to catch these sort of flaws prior to publication. This means flaws can be fixed, the claims made can be moderated and properly caveated, or publication can be prevented entirely so that research below a certain threshold of quality or rigour is not given the stamp of approval. (This helps readers know what's worth paying attention to and what isn't.) Research published via blogs and self-published reports don't go through academic peer review, and may fall below the standards of academic publishing. In the absence of peer review doing quality control, deeply flawed research, or deeply flawed interpretations of research, may propagate.

Even if there is say, a 30% chance that in the next 5 years even 30-40% of white collar jobs get replaced...that feels like a massive shock to high-income countries way of life, and a major shift in the social contract for the generation of kids who have got into massive debt for university courses only to find substantially reduced market opportunities. That requires substantial state action.

Rather than 30%, I would personally guess the chance is much less than 0.01% (1 in 10,000), possibly less than 0.001% (1 in 100,000) or even 0.0001% (1 in 1 million).

I agree with Huw that there's insufficient evidence to say that AI is causing significant or even measurable unemployment right now, and I'm highly skeptical this will happen anytime soon. Indeed, I'd personally guess there's a ~95% chance there's an AI bubble that will pop sometime within the next several years. So far, AI has stubbornly refused to deliver the sort of productivity or automation that has been promised. I think the problem is a fundamental science problem, not something that can be solved with scaling or incremental R&D. 

However, let's imagine a scenario where in a short time, say 5 years, a huge percentage of jobs get automated by AI — automation of white-collar work on a massive scale.^[1] 

What should governments' response be right now, before this has started to happen, or at least before there is broad agreement it has started to happen? People often talk about this as a gloomy, worrying outcome. I suppose it could turn out to be, but why should that be the default assumption? It would lead to much faster economic growth than developed countries are used to seeing. It might even be record-breaking, unprecedented economic growth. It would be a massive economic windfall. That's a good thing.^[2]

To be a bit more specific, when people imagine the sort of AI that is capable of automating white-collar work on the scale you're describing (30-40% of jobs), they also often imagine wildly high rates of GDP growth, ranging from 10% to 30%.^[3][4] The level of growth is supposed to be commensurate with the percentage of labour that AI can automate. I don't know about these specific figures, but the general idea is intuitive.

Surely passing UBI would become much easier once both a) unemployment significantly increased and b) economic growth significantly increased. There would be both a clear problem to address and a windfall of money with which to address it. By analogy, it would have been much harder for governments to pass stimulus bills in January 2020 in anticipation of covid-19. In March 2020, it was much easier, since the emergency was clear. But the covid-19 emergency caused a recession. What if, instead, it had caused an economic boom, and a commensurate increase in government revenues? Surely, then, it would have been even easier to pass stimulus bills. 

This is why, even if I accept the AI automation scenario for the sake of argument, I don't worry about the practical, logistical, or fiscal obstacles to enacting UBI in a hurry. Governments can send cheques to people on short notice, as we saw with covid-19. This would presumably be especially true if the government and the economy overall were experiencing a windfall from AI. The sort of administrative bottlenecks we saw in some places during in covid-19 — those could be solved by AI, since we're stipulating an unlimited supply of digital white-collar workers. Maybe there are further aspects to implementing UBI that would be more complicated than sending people cheques and that couldn't be assisted by AI. What would those be? 

The typical concerns raised over UBI are that it would be too expensive, that it would be poorly targeted (i.e. it would be more efficient to run means-tested programs), that it would discourage people from working, and that it would reduce economic growth. None of those apply to this scenario. 

If there's more complexity in implementing UBI that I'm not considering, surely in this scenario politicians would quickly become focused on dealing with that complexity, and civil servants (and AI workers) would be assigned to the task. As opposed to something like decarbonizing the economy, UBI seems like it could be implemented relatively quickly and easily, given a sudden emergency that called for it and a sudden windfall of cash. Part of the supposed appeal of UBI is its simplicity relative to means-tested programs like welfare and non-cash-based programs like food stamps and subsidized housing. So, if you're not satisfied with my answer, maybe you could elaborate on why you think it wouldn't be so easy to figure out UBI in a hurry.

As mentioned up top, I regard this just as an interesting hypothetical, since I think the chance of this actually happening is below 0.01% (1 in 10,000).  

1. ^{^}

   Let's assume, for the sake of argument, that the sort of dire outcomes hypothesized under the heading of AI safety or AI alignment do not occur. Let's assume that AI is safe and aligned, and that it's not able to be misused by humans to destroy or take over the world.

   Let's also assume that AI won't be a monopoly or duopoly or oligopoly but, like today, even open source models that are free to use are a viable alternative to the most cutting-edge proprietary models. We'll imagine that the pricing power of the AI companies will be put in check by competition from both proprietary and open source models. Sam Altman might become a trillionaire, but only a small fraction of the wealth created by AI will be captured by the AI companies. (As an analogy, think about how much wealth is created by office workers using Windows PCs, and how much of that wealth is captured by Microsoft or by PC manufacturers like HP and Dell, or components manufacturers like Intel.)

   I'm putting these other concerns aside in order to focus on labour automation and technological unemployment, since that's the concern you raised.

2. ^{^}

   The specific worries around AI automation people most commonly cite are about wealth distribution, and about people finding purpose and meaning in their lives if there's large-scale technological unemployment. I'll focus on the wealth distribution worry, since the topic is UBI, and your primary concern seems to be economic or material.

   Some people are also worried about workers being disempowered if they can be replaced by AI, at the same time that capital owners become much wealthier. If they're right to worry about that, then maybe it's important to consider well in advance of it happening. Maybe workers should act while they still have power and leverage. But it's a bit of a separate topic, I think, from whether to start implementing UBI now. Maybe UBI would be one of a suite of policies workers would want to enact in advance of large-scale AI automation of labour, but what's to prevent UBI from being repealed after workers are disempowered?

   For the sake of this discussion, I'll assume that workers (or former workers) will remain politically empowered, and healthy democracies will remain healthy.

3. ^{^}

   Potlogea, Andrei. “AI and Explosive Growth Redux.” Epoch AI, 20 June 2025, https://epoch.ai/gradient-updates/ai-and-explosive-growth-redux.

^{^}

Davidson, Tom. “Could Advanced AI Drive Explosive Economic Growth?” Coefficient Giving, 25 June 2021, https://coefficientgiving.org/research/could-advanced-ai-drive-explosive-economic-growth/.

Show all footnotes

Hence my initial mention of "high state capacity"? But I think it's fair to call abundance a deregulatory movement overall, in terms of, like... some abstract notion of what proportion of economic activity would become more vs less heavily involved with government, under an idealized abundance regime.

I guess it depends what version of abundance you're talking about. I have in mind the book Abundance as my primary idea of what abundance is, and in that version of abundance, I don't think it's clear that a politics of abundance would result in less economic activity being heavily involved with government. It might depend how you define that. If laws, regulations, or municipal processes that obstruct construction count as heavy involvement with the government, then that would count for a lot of economic activity, I guess. But if we don't count that and we do count higher state capacity, like more engineers working for the government, then maybe abundance would lead to a bigger government. I don't know.

I think you're right about why abundance is especially appealing to people of a certain type of political persuasion. A lot of people with more moderate, centrist, technocratic, socially/culturally less progressive, etc. tendencies have shown a lot of enthusiasm about the abundance label. I'm not ready to say that they now own the abundance label and abundance just is moderate, centrist, technocratic, etc. If a lot of emos were a fan of my favourite indie rock band, I wouldn't be ready to call it an emo band, even if I were happy for the emos' support.

There are four reasons I want to deconflate abundance and those other political tendencies:

1. It's intellectually limiting, and at least partially incorrect, to say that abundance is conceptually the same thing as a lot of other independent things that a lot of people who like abundance happen to also like.
2. I think the coiners and popularizers of abundance deserve a little consideration, and they don't (necessarily, wholeheartedly) agree with those other political tendencies. For instance, Ezra Klein has, to me, been one of the more persuasive proponents of Black Lives Matter for people with a wonk mindset, and has had guests on his podcast from the policy wonk side of BLM to make their case. Klein and Thompson have both expressed limited, tepid support for left-wing economic populist policies, conditional on abundance-style policies also getting enacted.
3. I'm personally skeptical of many of the ideas found within those other political tendencies, both on the merits and in terms of what's popular or wins elections. (My skepticism has nothing to do with my skepticism of the ideas put forward in the book Abundance, which overall I strongly support and which are orthogonal to the ideas I'm skeptical of.)
4. It's politically limiting to conflate abundance and these other political tendencies when this isn't intellectually necessary. Maybe moderates enjoy using abundance as a rallying cry for their moderate politics, but conflating abundance and moderate politics makes it a polarized, factional issue and reduces the likelihood of it receiving broad support. I would rather see people try to find common ground on abundance rather than claim it for their faction. Gavin Newsom and Zohran Mamdani are both into abundance, so why can't it have broad appeal? Why try to make it into a factional issue rather than a more inclusive liberal/left idea?

Edit: I wrote the above before I saw what you added to your comment. I have a qualm with this:

But, uh, this is the EA Forum, which is in part about describing the world truthfully, not just spinning PR for movements that I happen to admire.  And I think it's an appropriate summary of a complex movement to say that abundance stuff is mostly a center-left, deregulatory, etc movement.

I think it really depends on which version of abundance you're talking about. If you're talking about the version in the book Abundance by Ezra Klein and Derek Thompson, or the version that the two authors have more broadly advocated (e.g. on their press tour for the book, or in their writing and podcasts before and after the book was published), then, no, I don't think that's an accurate summary of that particular version of abundance. 

If you're referring to the version of abundance advocated by centrists, moderates, and so on, then, okay, it may be accurate to say that version of abundance is centrist, moderate, etc. But I don't want to limit how I define to "abundance" to just that version, for the reasons I gave above.

I don't think it makes sense to call it "spin" or "PR" to describe an idea in the terms used by the originators of that idea, or in terms that are independently substantively correct, e.g. as supported by examples of progressive supporters of abundance like Mamdani. If your impression of what abundance is comes from centrists, moderates, and so on, then maybe that's why you have the impression that abundance simply is centrist, moderate, etc. and that saying otherwise is "untruthful" or "PR". There is no "canonical" version of abundance, so to some extent, abundance just means what people who use the term want it to mean. So, that impression of abundance isn't straightforwardly wrong. It's just avoidably limited.

Imagine someone complaining -- it's so unfair to describe abundance as a "democrat" movement!!  That's so off-putting for conservatives -- instead of ostracising them, we should be trying to entice them to adopt these ideas that will be good for the american people!  Like Montana and Texas passing great YIMBY laws, Idaho deploying modular nuclear reactors, etc.  In lots of ways abundance is totally coherent with conservative goals of efficient government services, human liberty, a focus on economic growth, et cetera!!

To the extent people care what Abundance says in deciding what abundance is, one could quote from the first chapter of the book, specifically the section "A Liberalism That Builds", which explicitly addresses this topic.