That's fair; upon re-reading your comment it's actually pretty obvious you meant the conditional probability, in which case I agree multiplying is fine.
I think the conditional statements are actually straightforward - e.g. once we've built something far more capable than humanity, and that system "rebels" against us, it's pretty certain that we lose, and point (2) is the classic question of how hard alignment is. Your point (1) about whether we build far-superhuman AGI in the next 30 years or so seems like the most uncertain one here.
In general I quite like this post, I think it elucidates some disagreements quite well.
Thanks!
Iâm not sure it represents the default-success argument on uncertainty well.
I haven't tried to make an object-level argument for either "AI risk is default-failure" or "AI risk is default-success" (sorry if that was unclear). See Nate's post for the former.
Re your argument for default-success, you only need to have 97% certainty for 1-4 if every step was independent, which they aren't.
I do agree that discussion is better pointed to discussing this evidence than gesturing to uncertainty
Agreed.
Thanks for the post, it was an interesting read!
Responding to one specific point: you compare
Community members delegate to high-quality research, think less for themselves but more people end up working in higher-impact causes
to
Community members think for themselves, which improves their ability to do more good, but they make more mistakes
I think there is actually just one correct solution here, namely thinking through everything yourself and trusting community consensus only insofar as you think it can be trusted (which is just thinking through things yourself on the meta-level).
This is the straightforwardly correct thing to do for your personal epistemics, and IMO it's also the move that maximizes overall impact. It would be kind of strange if the right move was for people to not form beliefs as best they can, or to act on other people's beliefs rather than their own?
(A sub-point here is that we haven't figured out all the right approaches yet so we need people to add to the epistemic commons.)
FWIW I fit that description in the sense that I think AI X-risk is higher probability. I imagine some / most others at LTFF would as well.