Comments
This talk has one approach.
More from the author
Curated and popular this week
61
This is a crosspost from the new Animal Welfare Alignment Newsletter by Anima International. You can subscribe on Substack if you are interested in following these efforts. Audio reading also available on Substack.
The goals of this post are to:
1. Raise a question I see as crucially important to the goal of aligning AI to animal welfare...
Hello! I'm Justin Portela. I got hired by GWWC to make YouTube videos after AI in Context did such a kickass job.
My channel is using that same cinematic, high-production value beauty to talk about everything in the EA universe that isn't AI.
...
I used AI to fix transcription errors, rerrarange the ideas, and suggest tweaks to the title and some sentences.
Three of the most exciting projects to come out of EA in recent years are, in a vague sense, CEA spinouts:
* Kairos is directly a spinout of CEA and now handles most support for university AI safety groups. Basically everyone I've found who knows them is really excited about what they do
* NEST is an opinionated ideas-fi...
Recent opportunities to take action
I think most probabilistic estimates are subjective probability estimates. There are no complicated math models behind them usually.
Some people do make models, but then make subjective probability estimates. The math is typically not that complicated for these models, often just multiplying different probabilities together (which is imo not a good class of models for this kind of problem).
My guess would be that even some of the people who make models have different probability estimates for human extinction than the one that the model spits out, because they realize that their models have flaws and try to correct for that.
So the prediction experts made are all pureb"subjective" predictions? I think there are some logical thinking/arguments or maybe like fermi estimation to explain how he estimates the number unless it's mostly intuition.
[This article](https://slatestarcodex.com/2013/05/02/if-its-worth-doing-its-worth-doing-with-made-up-statistics/) explores why it is useful to work with subjective, "made-up" statistics.
My own view hinges on the following:
These factors lead me to think we have significantly worse than even odds (that is, <50%) of surviving this century.
I'm also quite interested in how these estimates are being made, so can I ask you for more detail about how you got your estimate?
In particular, I'm interested in the "chain of events" involved. AI extinction involves several consecutive speculative events. What are your estimates for the following, conditional on the previous steps occurring?
Did your >50% estimate come from reasoning like this about each step?
I think 1 is >95% likely. We're in an arms race dynamic for at least some of the components of AGI. This is conditional on us not having been otherwise wiped out (by war, pandemic, asteroid, etc).
I think 2 and 3 are the wrong way to think about the question. Was humankind "motivated to conquer" the dodo? Or did we just have a better use for its habitat, and its extinction was just a whoopsie in the process?
When I say "motivated to", I don't mean that it would be it's primary motivation. I mean that it has motivations that, at some point, would lead to it having "perform actions that would kill all of humanity" as a sub-goal. And in order to get to the point where we were dodo's to it, it would have to disempower humanity somehow.
Would you prefer the following restatement, each conditional on the previous step:
And then either:
4a: The process of disempowering humanity involves wiping out all of humanity
Or
4b: After successfully disempowering humanity with some of humanity still intact, the AI ends up wiping out the rest of humanity anyway