This is full, but it's worth getting people to subscribe for the future
Hmm... often I think it is nice to have a standard term for a phenomenon so that people don't have to figure out how to express a certain concept each time and then hope that everyone else can follow. Language also has the advantage that insofar as we convince people to adopt our language, we draw them into our worldview.
This should really be a Wiki page instead since these lists (I even made one myself in the past) always become outdated.
This is a really challenging situation - I could honestly see myself leaning either way on this kind of scenario. I used to lean a lot more towards saying whatever I thought was true and ignoring the consequences, but lately I've been thinking that it's important to pick your battles.
I think the key sentence is this one - "On many subjects EAs rightfully attempt to adopt a nuanced opinion, carefully and neutrally comparing the pros and cons, and only in the conclusion adopting a tentative, highly hedged, extremely provisional stance. Alas, this is not such a subject."
What seems more important to me is not necessarily these kinds of edge cases, but that we talk openly about the threat potentially posed. Replacing the talk with a discussion about cancel culture instead seems like it could have been a brilliant Jiu Jitsu move. I'm actually much more worried about what's been going on with ACE than anything else.
Thanks, that was useful. I didn't realise that his argument involved 1+2 and not just 1 by itself. That said, if the hinge of history was some point in the past, then that doesn't affect our decisions as we can't invest in the past. And perhaps it's a less extraordinary coincidence that the forward-looking hinge of history (where we restrict the time period from now until the end of humanity) could be now, especially if in the average case we don't expect history to go on much longer.
I've never found Will's objections to the hinge of history argument persuasive. Convincing me that there was a greater potential impact in past times than I thought, ie. that it would have been very influential to prevent the rise of Christianity, shouldn't make me disbelieve that arguments that AI or bio risks are likely to lead to catastrophe in the next few decades if we don't do anything about it. But maybe I just need to reread the argument again.
DNA engineering has some positive points, but imagine the power that having significant control its citizens personalities would give the government. That shouldn't be underestimated.
The real hinge here is how much we should expect the future to be a continuation of the past and how much we update based on our best predictions. Given what we know about about existential risk and the likelihood that AI will dramatically change our economy, I don't think this idea makes sense in the current context.
I agree that such a system would be terrifying. But I worry that its absence would be even more terrifying. Limited surveillance systems work decently for gun control, but when we get to the stage where someone can kill tens of thousands or even millions instead of a hundred I suspect it'll break down.
Thanks for posting such a detailed answer!