Infinite Player
I think that by far the less intuitive thing about AI X-risk is why AIs would want to kill us instead of doing what they would be "programmed" to do.
I would give more importance to that part of the argument than the "intelligence is really powerful" part.
I think the bigger question is why haven't we found other species elsewhere in the universe.
Then I see the question about whether they'll kill us or not as a different one.
I really liked the axis that you presented and the comparision between a version of the community that is more cause oriented vs member oriented.
The only caveat that I have is that I don't think we can define a neutral point in between them that allows you to classify communities as one type or the other.
Luckily, I think that is unnecesary because even though the objective of EA is to have the best impact in the world and not the greatest number of members, I think we all think the best decision is to have a good balance between cause and member oriented. So the question that we should ask is should EA be MORE big tent, weird, or do we have a good balance right now?
And to achieve that balance we can be more big tent in some aspects, moments and orgs and weirder in others.
I strongly agree with you and, as someone who thinks alignment is extremely hard (not just because of the technical side of the problem, but due the human values side too), I believe that a hard pause and studying how to augment human intelligence is actually the best strategy.
Wait, I don't understand. Are 63.6% or 76.6% of respondents left-leaning? And 69.58% or 79.81% non-religious?
Have important "crying wolf" cases have happen in real life? About societal issues? I mean, yeah, it is a possibility but the alternatives seem so much worse.
How do we know when we are close enough to the precipice for other people to be able to see it and to ask to stop the race to it? General audiences lately have been talking about how surprised they are about AI so it seems like perfect timing for me.
Also, if people get used to benefit and work in narrow and safe AIs they could put themselves against stopping/ slowing them down.
Even if more people could agree on decelerate in the future it would take more time to stop/ going really slow with more stakeholders going at a higher speed. And of course, after that we would be closer to the precipice that if we started the deceleration before.
I doubt this:
the AGI doom memeplex has, to some extent, a symbiotic relationship with the race toward AGI memeplex
I mean, if you say it could increase the amount of people working in capabilities at first I would agree, but it probably increases a lot more the amount of people working on safety and wanting to slow/ stop capabilities research, which could create legislation and at the end of the day increase time until AGI.
In respect of the other cons of the doom memeplex I kinda agree to a certain extent but I don't see them come even close to the pros of potentially having lots of people actually taking the problem very seriously.
Where do you get this from?
Also, this:
means p(doom) <~ 0.001