Comments
Let’s imagine it’s November 2031. We’re both still here, AI is much more capable than it is today, but governments are governing, civilisation is still (mostly) civilised, and there has been no catastrophic loss of control. It just kind of didn’t happen.
What happens to the AI safety field then?
I expect people who currently believe catastrophe could happen within two or three years will start saying five or ten years. Then 15. Then ‘who cares if we’re a few decades off; we’re talking about human extinction!’ Meanwhile, people who have spent years warning that we could all die soon might remind everyone that they only said could.
For people who are very scared, AI x-risk seems to have become an organising principle of their lives. Careers, financial decisions (not to mention funding decisions), whether to have children, whether to move to SF - influenced by the belief that humanity might have only a few years left.
If that ends up not being the case, the overriding temptation might be to say ‘it’s still just around the corner and, given the stakes, it’s dumb of you to expect a clear cut off point’, keep sounding the alarm for as long as funding lasts, and then… transition into something else.
…
I’d like to see more people putting numbers and dates against their predictions. Then let’s revisit those predictions in five years. EA prides itself on intellectual honesty and updating beliefs in response to evidence etc. Will those standards be applied as much to AI safety organisations as they are to most everything else?
At this point it’s probably obvious that I’m having a hard time understanding the threats are as dire as some people say. But I can’t make a serious plaidoyer for or against a belief in catastrophic AI risk, because I’m uninformed. ‘Huh, tell us something we don’t know, given this post’, some might say. Fair enough. I’ve done a couple of BlueDot courses, read around the subject, and I’m cognisant of how much the experts disagree, which makes it hard to feel conviction in either direction. Yet I hope you’ll still consider it a worthy question, no matter who’s asking. A necessary question.
What would it take for the people who are both informed and scared, to change their minds?
21
I think this question is pretty hard for me to answer because it's hard for me to imagine what the future would look like if we have AIs that are actually much more capable than today but the world mostly stays normal. I think it's much more plausible to me that I'm wrong because AIs don't become much capable, than because they do become much more capable and nothing external much changes.
What about AI becoming unsafe at a normal you-can-be-sued-for-this level strongly enough that you don't really get much forward progress because nobody can release or use their models, and this drives some combo of an AI financial collapse and a shift towards specialized models/harnesses/other anti-bitter lesson stuff? Does this scenario put too much trust in the AI companies?
Right, so to be a bit more concrete, maybe you're imagining that after HuggingFace 3.0 the push for generalized AIs stall out and we see some impressive specialized innovations like (say) self-driving cars and specialized biology models but no more general advances? Seems plausible.
Such a world I think look a lot like some combination of my #1 and #3, except with probably less available funding for AI safety than those worlds (though likely still higher than 2025 and 2026).