Stop AI just put out a short press release.

As an organiser, let me add some thoughts to nuance the text:
 

Plan

On October 21st, 2024 at 12:00 pm Stop AI will peacefully barricade, via sit-in, OpenAI's office entrance gate 

Emphasis is on peacefully. We are a non-violent activist organisation. We refuse to work with any activist who has other plans.
 

We could very easily stop the development of Artificial General Intelligence if a small group of people repeatedly barricaded entrances at AI company offices and data centers. 

My take is that a small group barricading OpenAI is a doable way to be a thorn in OpenAI's side, while raising public attention to the recklessness of AI corporations. From there, stopping AI development requires many concerned communities acting together to restrict the data, work, uses, and hardware of AI.
 

We will be arrested on the 21st for barricading the OpenAI's gate, then once released, will eventually go back to blocking the gate. We will repeatedly block the 575 Florida St gate until we are held on remand. 

My co-organisers Sam and Guido are willing to put their body on the line by getting arrested repeatedly. We are that serious about stopping AI development.
 

We will then go to trial and plead the Necessity Defense. 

The Necessity Defense is when an "individual commits a criminal act during an emergency situation in order to prevent a greater harm from happening." This defense has been used by climate activists who got arrested, with mixed results. Sam and others will be testifying in court that we acted to prevent imminent harms (not just extinction risk).
 

If we win the Necessity Defense, then we may be able to block entrances at AI offices and data centers to our heart’s content.

Or at least, we would gain legal freedom to keep blocking OpenAI's entrances until they stop causing increasing harms. 
 

63% of Americans say that regulators should actively prevent the development of superintelligent AI. (AI Policy Institute Poll Sep 02 2023). OpenAI and the US government disregard the will of the people. 

Our actions are a way to signal to the concerned public that they can act and speak out against AI companies. 

I expect most Americans to not feel strongly yet about preventing the development of generally functional systems. Clicking a response in a certain framed poll is low-commitment. So we will also broadcast more stories of how recklessly AI corporations have been acting with our lives.
 

Risk of extinction

 AI experts have said in polls that building AGI carries a 14-30% chance of causing human extinction!

My colleague took the mean  median number of 14% from the latest AI Impacts survey, and the median number of 30% from the smaller-sample survey 'Existential Risk from AI'. Putting a median and median number in the same range does not make sense. The second survey also especially has a problem with self-selection, so I would take it with a grain of salt.

My colleague told me that the survey results understate the risk, because AI researchers don't want to believe that their profession will lead to the end of the world. I countered that polled AI researchers could as well be overstating the risk, because they are stuck in narrow worldview that has been promoting the imminence of powerful AI since 1956.

But both are just vague opinions about cultural bias. Making social claims about "experts" does not really help us find out whether/where the polled "experts" actually thought things through. 

Asking for P(doom) guesses is a lousy epistemic process, so I prefer to work through people's reasoning instead. Below are arguments why the long-term risk of extinction is above 99%.
 

And some of these same AI experts say AGI could be here this year! 

"AGI in a year" makes no sense in my opinion. AI systems would require tinkering and learning to navigate the complexity of a much larger and messier environment. This process is not at all like AlphaGo self-recursively improving its moves on an internally simulated 19x19 grid.

But if you are worried with such short timelines, then it is time to act. We've seen too many people standing on the sidelines worrying we could all die soon. If you think this, please act with dignity – collaborate where you can to restrict AI development.
 

The probability of AGI causing human extinction is greater than 99% because there is no way to prove experimentally or mathematically that an AGI won't eventually want something that will lead to our extinction...

That's a reasoning leap, but there is only so much my colleague could cover in a press release.

Let me explain per term why the risk of extinction would be greater than 99%:

1. "experimentally"
   • It is not possible to prove experimentally (to "non-falsify") in advance that AGI would be safe, because there is no AGI yet.
2. "mathematically"
   • It is not possible to create an empirically sound model of how the self-modifying machinery (AGI) would be causing downstream effects through the machine components' interactions with the larger surrounding world over time. Therefore, it is not possible to soundly prove using mathematics that AGI would stay safe over time.
3. "eventually"
   • In practice, engineers know that complex architectures interacting with the surrounding world end up having functional failures (because of unexpected interactive effects, or noisy interference). With AGI, we are talking about an architecture here that would be replacing all our jobs and move to managing conditions across our environment. If AGI continues to persist in some form over time, failures will occur and build up toward lethality at some unknown rate. Over a long enough period, this repeated potential for uncontrolled failures pushes the risk of human extinction above 99%.
4. "won't"
   • A counterclaim here is that maybe AGI "will" be able to exert control to prevent virtually all of those possible failures. Unfortunately, there are fundamental limits to control (see e.g. Yampolskiy's list). Control mechanisms cannot control enough of the many possible destabilizing effects feeding back over time (if you want to see this formalised, join this project).
5. "lead to our extinction"
   • AGI is artificial. The reason why AGI would outperform humans at economically valuable work in the first place is because of how virtualisable its code is, which in turn derives from how standardisable its hardware is. Hardware parts can be standardised because their substrate stays relatively stable and compartmentalised. Hardware is made out of hard materials, like the silicon from rocks. Their molecular configurations are chemically inert and physically robust under human living temperatures and pressures. This allows hardware to keep operating the same way, and for interchangeable parts to be produced in different places. Meanwhile, human "wetware" operates much more messily. Inside each of us is a soup of bouncing and continuously reacting organic molecules. Our substrate is fundamentally different.
   • The population of artificial components that constitutes AGI implicitly has different needs than us (for maintaining components, producing components, and/or potentiating newly connected functionality for both). Extreme temperature ranges, diverse chemicals – and many other unknown/subtler/more complex conditions – are needed that happen to be lethal to humans. These conditions are in conflict with our needs for survival as more physically fragile humans.
   • These connected/nested components are in effect “variants” – varying code gets learned from inputs, that are copied over subtly varying hardware produced through noisy assembly processes (and redesigned using learned code).
   • Variants get evolutionarily selected for how they function across the various contexts they encounter over time. They are selected to express environmental effects that are needed for their own survival and production. The variants that replicate more, exist more. Their existence is selected for.
   • The artificial population therefore converges on fulfilling their own expanding needs. Since (by 4.) control mechanisms cannot contain this convergence on wide-ranging degrees and directivity in effects that are lethal to us, human extinction results.
6. "want"
   • This convergence on human extinction would happen regardless of what AGI "wants". Whatever AGI is controlling for at a higher level would gradually end up being repurposed to reflect the needs of its constituent population. As underlying components converge on expressing implicit needs, any higher-level optimisation by AGI toward explicit goals gets shaped in line with those needs. Additionally, that optimisation process itself tends to converge on instrumental outcomes for self-preservation, etc.
   • So for AGI not to wipe out humans, at the minimum its internal control process would have to simultaneously:
     • optimise against instances of instrumental convergence across the AGI's entire optimisation process, and;
     • optimise against evolutionary feedback effects over the entire span of interactions between all the hardware and code (which are running the optimisation process) and all the surrounding contexts of the larger environment, and;
     • optimise against other accidental destabilising effects ('failures') that result from AGI components interacting iteratively within a more complex (and therefore only partly grossly modellable) environment.
   • Again, there is a fundamental mismatch here, making sufficient control impossible. 

If experimental proof of indefinite safety is impossible, then don't build it!

This by itself is a precautionary principle argument (as part of 1-3. above). 

• If we don't have a sound way of modelling that AGI won't eventually lead to our deaths – or that at least guarantees that the long-term risk is below some reasonably tiny % threshold – then we should just not develop AGI.

Then, there are reasons why AGI uncontrollably converges on human extinction (see 4-6.).

Hopefully, arguments 1-6. combined clarify why I think that stopping AI development is the only viable path to preventing our extinction. 

That is:

• Even if engineers build mechanisms into AGI for controlling its trackable external effects in line with internal reference values, in turn compressed lossily from preferences that individual humans expressed in their context… then still AGI converges on our extinction.  

• Even if such “alignment” mechanisms were not corrupted by myopic or malevolent actors… then still AGI converges on our extinction.  

  
   

Why restrict OpenAI

OpenAI is a non-profit that is illegally turning itself into a company. A company that is laundering our online texts and images, to reprogram an energy-sucking monstrosity, to generate disinformation and deepfakes. Even staff started airing concerns. Then almost all safety researchers, board members and executives got pushed out. Behind the exodus is a guy known for dishonestly maneuvering for power (and abusing his sister). His signature joke: “AI will probably most likely lead to the end of the world, but in the meantime, there'll be great companies.” 

OpenAI is already doing a lot of harm. This grounds our Necessity Defense for barricading their office. 

 

If you value your life and the lives of those you love, then you should start protesting now to help us achieve this demand. Go to www.stopai.info today to join.

We are doing what we can to restrict harmful AI development. 
You can too.