I think accelerating AI is justified largely because of the enormously positive impacts I expect it to bring to billions of people alive today: faster medical developments, dramatically increased economic prosperity, and much greater product variety from technological innovation.
Some have asked me: why do I focus on these short-term impacts of accelerating AI to justify my pro-acceleration view, rather than on the impacts that might occur over the upcoming billions of years?
The first thing worth pointing out about this question is that, at least in my experience, it almost never comes up outside of debates about AI. In virtually every other area of human decision-making, people generally accept without much argument that the very-long-term consequences of our actions are extremely difficult to predict. Because of this extreme uncertainty, most people intuitively recognize that we should place less weight on projected very-long-term effects, since we simply cannot know with any confidence what those effects will actually turn out to be.
When policymakers debate housing regulations, or when patients decide whether to undergo a medical procedure, or when nations consider going to war, almost nobody would accept an argument structured like this: "We should willingly accept devastating costs over the coming decades, like entering a catastrophic military conflict, because according to my calculations, doing so will produce enormous benefits over the course of the next billion years."
Most people rationally reject arguments like this, and their rejection is not merely rooted in impatience or in a callous disregard for future generations. Instead, people reject such arguments because they recognize that our ability to predict consequences over such enormous timescales is essentially nonexistent. Even in a relatively well-studied area like housing policy, where researchers have accumulated extensive data and conducted rigorous analysis, experts still reasonably disagree about what effects a given policy will have over the next twenty or thirty years. If we struggle to forecast impacts over mere decades in a data-rich field, then claiming to know what effects a policy will have over billions of years is simply not credible.
Given all of this, the reason I focus on short-term impacts when defending AI acceleration is fundamentally the same reason I would focus on short-term impacts when evaluating pretty much any other policy question: short-term consequences are the only consequences we can predict with any meaningful degree of reliability. When it comes to the long-term effects of speeding up or slowing down AI development, our uncertainty is profound. In light of this uncertainty, I believe we should be deeply reluctant to accept policies that impose massive, concrete harms on people alive today in exchange for hypothetical benefits that are not only extremely speculative but also would not fully materialize for billions of years.
Inevitably, many people will find this response unsatisfying. A significant number of thinkers, particularly effective altruists, believe that AI represents a fundamentally different kind of issue: one that requires a distinct analytical framework rather than the standard tools we apply to other policy questions. The key claim these thinkers make is that AI development is uniquely connected to the possibility of human extinction. Unlike housing policy, medical decisions, or even war, they argue, AI poses a direct threat to humanity's continued existence. And according to them, risks to human existence carry an extraordinary moral weight that ordinary risks do not.
The intellectual foundation for this position can be found in Nick Bostrom's influential essay titled Astronomical Waste. In that essay, Bostrom argues that preventing human extinction is far more valuable than accelerating technological progress, even if that technological progress would dramatically extend human lifespans and greatly improve human wellbeing. His reasoning proceeds as follows: speeding up the development of advanced technology might, at best, save billions of human lives. That sounds like an enormous benefit, and in ordinary terms it would be. But consider what happens if humanity goes extinct. Extinction would not merely end the lives of people currently alive; it would permanently foreclose the possibility of a future in which humans colonize space and spread across the cosmos. Bostrom estimates that such a future could eventually support far more than 10^23 human lives. When you compare 10^23 to a few billion, the numbers are not even close. Therefore, Bostrom concludes, even a minuscule reduction in the probability of extinction will always deliver more expected value than saving billions of lives through faster technological development.
Why do I remain unpersuaded by this line of reasoning?
My first and most important objection is that Bostrom's original argument was specifically about risks that threatened to permanently eliminate all intelligent life on Earth. This assumption makes the most sense when we consider a scenario like a massive asteroid hurtling toward our planet. If such an asteroid were to collide with Earth, there is a scientifically credible basis for believing that the impact would not only kill every human being, but would also eliminate the opportunity for complex life to evolve again and eventually spread throughout the universe. Under those conditions, it is reasonable to conclude that successfully deflecting the asteroid would meaningfully increase the probability that Earth-originating life will eventually colonize the cosmos.
However, while this reasoning is sound when applied to a catastrophic asteroid impact, it becomes much less convincing when applied to artificial intelligence. The AI scenario differs fundamentally from the asteroid scenario. Suppose we assume, for the sake of argument, that advanced AI systems do eventually destroy humanity. What happens next? Unlike an asteroid impact, which leaves behind a lifeless or barely habitable world, the AI systems that destroyed humanity would presumably continue to exist and function. These AI systems would likely go on to build their own civilization, and that AI civilization could itself eventually expand outward and colonize the cosmos. Understood in this way, AI does not actually pose a risk of astronomical catastrophe in Bostrom's sense. Instead, AI poses what we might call a catastrophe of replacement: artificial intelligence would take the place that humans might otherwise have occupied, but the broader potential for Earth-originating intelligence to spread across the universe would remain intact.
Now, you might still reasonably be very concerned about such a replacement catastrophe. I myself share that concern and take the possibility seriously. But it is crucial to keep the structure of the original argument clearly in mind. If AI systems replace humanity, that outcome would undoubtedly be an absolute disaster for the eight billion human beings currently alive on Earth. However, it would be a localized, short-term disaster rather than an astronomical one. Bostrom's argument, strictly interpreted, no longer applies to this situation. The reason is that the risk is confined to the present generation of humans: the question at stake is simply whether the eight billion people alive today will be killed or allowed to continue living. Even if you accept that killing eight billion people would be an extraordinarily terrible outcome, it does not automatically follow that this harm carries the same moral weight as a catastrophe that permanently eliminates the possibility of 10^23 future lives.
Why does this distinction matter? It matters because once we recognize that the risk from AI is fundamentally about its potential to kill eight billion individual human beings, we must then weigh that risk against the potential for AI to save and dramatically improve billions of human lives through accelerated medical developments and increased economic prosperity. This comparison becomes especially important when we notice a crucial symmetry in the arguments about AI danger. Virtually every proposed mechanism by which AI systems might cause human extinction relies on the assumption that these AI systems would be extraordinarily capable, productive, or technologically sophisticated. But this same assumption implies, with equal force, that AI systems could benefit humanity just as profoundly as they could harm us.
Consider, for example, the extinction scenario that Eliezer Yudkowsky has described. His scenario depends on the premise that AI systems could quickly develop advanced molecular nanotechnology capable of matching or even surpassing the sophistication of biological systems. But technology that powerful could just as easily be directed toward curing aging or creating unprecedented material abundance for billions of people as it could be directed toward destroying us.
These considerations suggest that delaying the development of beneficial artificial intelligence carries genuinely enormous costs that cannot simply be waved away by pointing to supposedly overwhelming astronomical consequences on the side of caution. To be clear, I am not arguing that extinction risk is irrelevant or that we should never accept any slowdown in AI progress for safety reasons. If delaying AI development by one year would reduce the probability of human extinction by ten percentage points, that tradeoff would seem clearly worthwhile to me. However, the calculus changes substantially when we consider different parameters. Delaying AI development by an entire decade in order to reduce extinction risk by a single percentage point no longer seems like a worthwhile tradeoff. The reason is that a decade of delayed progress would mean that nearly a billion people will die from diseases and age-related decline who might otherwise have been saved by the rapid medical advances that AI could enable. Those billion people would have gone on to live much longer, healthier, and more prosperous lives.
Once we recognize these tradeoffs, the question of whether to delay AI development becomes a question that depends on specific parameters that we must estimate through empirical investigation. It is not a question that we can resolve simply by writing philosophy papers that declare caution to be overwhelmingly more valuable than acceleration, based on nothing more than inserting astronomically large numbers into one side of the equation. Instead, we must actually do the hard work of figuring out the relevant empirical facts. How much reduction in existential risk are we actually achieving by delaying AI development? Is there some alternative approach that could achieve the same level of risk reduction without imposing enormous costs measured in billions of human lives lost or diminished?
After spending the last several years investigating these questions, I have found myself siding with those who favor accelerating AI development. This position does not reflect total disregard for future generations on my part. Rather, I arrived at this position because I came to believe that the arguments in favor of pausing AI development rest on genuinely weak empirical foundations. Not only do I think the probability of human extinction directly caused by AI is very low, I also think that delaying AI development would not meaningfully lower this risk.
When we look at the history of technology, we find that the way technologies become safer is not by pausing their development and spending extended periods thinking abstractly about how to make them less dangerous. Nor do technologies typically become much safer through narrow experimentation confined to laboratory environments. Consider how planes, automobiles, electrical systems, elevators, anesthesia, and many other important technologies actually became safer over time. In each of these cases, safety improvements came about because society gained collective experience with the technology deployed at scale in the real world. This widespread deployment allowed us to observe and document the problems that arose when ordinary people used these technologies under real conditions. We could then identify solutions to those problems and implement fixes in subsequent versions of the technology. This iterative process of improvement depends fundamentally on the ongoing development and deployment of the technology in question. Simply halting development would mostly just interrupt this beneficial process rather than providing meaningful safety gains.
My current understanding is that AI development works the same way. This understanding appears to be supported by the improvements we have already observed in frontier LLMs over the past several years. These models have progressively gotten better at following what users actually want, and they have become more reliable at staying within the safety guidelines they have been given. By all available evidence, current AI models appear to be genuinely ethical and thoughtful in the vast majority of user interactions. I see no empirical basis for believing that this apparent alignment is actually an elaborate deceptive performance designed to manipulate us into complacency. Pretty much the only support I can find for such a view is the bare logical possibility that deceptive alignment could occur. But logical possibility alone, without supporting evidence, is not a strong foundation for policy.
Of course, I want to acknowledge that my argument here so far does not address every way in which pausing or accelerating AI development might produce meaningful long-term consequences. Many people have pushed back on my position by arguing that human extinction would still constitute an astronomical catastrophe even if AI systems survived and thrived, because they believe AI systems would lack consciousness and therefore lack moral value. In the words of Bostrom, we might end up with “a Disneyland without children”. I personally find this line of objection to be based on a deep confusion about consciousness. It assumes, without justification, that the biological substrate has a special quality producing moral value that will not be replicated in computational substrates, even when AIs become highly sophisticated, intelligent agents. But even if we were to grant this assumption for the sake of argument, the objection remains incomplete.
The relevant question is not simply whether delaying AI development would extend the period of time during which biological organisms maintain control over the world. You would need to show that you are not merely delaying the inevitable transition to an AI-controlled world. You would need to show that you are durably setting us on a better trajectory whose consequences will not simply wash out and become irrelevant over the very long run. That is, you would need to establish that delaying AI produces some permanent and predictable effect on the far future. This is an extraordinarily strong claim, and it is the kind of claim that most people would view with deep skepticism if it were made about other policy domains, such as housing policy, healthcare policy, or decisions about whether to go to war.
Other people have offered a different defense of delay, arguing that we should slow down AI development based on a general principle that we ought to exercise greater caution whenever we are approaching events with potentially enormous consequences. I am skeptical of this argument because I cannot identify any historical precedent where this reasoning was clearly vindicated in hindsight. Consider the Industrial Revolution as an example. Would it have been better for humanity in the long run if Britain had approached industrialization with far greater caution, implementing extensive controls and moving much more slowly? I do not see any reason to believe that would have been true. Furthermore, caution at the societal level is not a costless or neutral stance. Implementing societal caution typically requires top-down regulation and constraint, which in turn necessitates coercion and the restriction of individual freedom to innovate and experiment. Establishing such a regime of restriction does not seem like an obvious default position that we should simply adopt as our starting point.
If anything, my own inclination is toward a presumption of liberty, not caution. I favor allowing human beings the freedom to pursue innovation, which has historically been the force that lifted billions of people out of poverty, extended human lifespans through medical advances, and made daily life more comfortable and secure. The principle that we should always default to greater caution does not seem clearly justified by historical evidence. In many situations throughout history, the right course of action has been to embrace less caution rather than more.
Taken literally, "accelerationist" implies that you think the technology isn't currently progressing fast enough, and that some steps should be taken to make it go faster. This seems a bit odd, because one of your key arguments (that I actually agree with) is that we learn to adapt to technology as it rolls out. But obviously it's harder to adapt when change is super quick, compared to gradual progress.
How fast do you think AI progress should be going, and what changes should be made to get there?
Not trying to answer on the author's behalf, but it seems relatively clear to me that differential development is possible here: so far most advancements in science seem to have come from biological applications like AlphaFold that are distinct from the LLMs that have created most problems both in the eyes of "doomers" and in the eyes of people warning about current non-extinction dangers. Therefore the development of beneficial tools can in theory be accelerated while the development of LLMs is slowed down.
For what it's worth, this isn't my view. I think AlphaFold will have a much smaller effect on human health and wellbeing than general-purpose digital agents that can substitute for human workers across a variety of jobs.
Medical progress -- and economic progress more generally -- relies on building out extensive infrastructure for the discovery, development, manufacturing, distribution and delivery of innovations. For example, more spending on medical R&D in 1925 would not have led to widespread MRI machines, because creating MRI machines required building complementary industries, such as large-scale helium liquefaction plants, that would not have arisen through R&D alone. For similar reasons, I predict that better medical AI alone would not be sufficient to reverse aging, cure cancer, or prevent Alzheimer's.
In fact, I think the issue here is more fundamental than you might think: the very reason EAs are worried about general-purpose digital AI agents arises directly from the fact that these agents would be the most useful for accelerating technological progress. Their utility is precisely what makes them risky. You can't eliminate the danger without making them less useful. The two things are intrinsically linked.