This is really strongly giving off suspicious convergence vibes, especially "AI safety means you don't have to choose a cause".

Also, "AI is better than us" is kind of scary religious talk. It sounds like we are worshipping a god and trying to summon it :)

I really appreciate the sentiment behind this - I get the sense that working on AI safety can feel very doom-y at times, and appreciate any efforts to alleviate that mental stress.

But I also worry that leaning into these specific reasons may lead to intellectual blindspots. E.g., Believing that aligned AI will make every other cause redundant leads me to emotionally discount considerations such as temporal discount rate or tractability. If you can justify your work as a silver bullet, then how much longer would you be willing to work on that, even when it seems impossible? Where does one draw the line?

My main point here is that these reasons can be great motivators, but should only be called upon after someone has intellectually mapped out the reasons why they are working on AI and what would need to change for them to stop working on it.

On some level, I get all of this, and try to think about it to inspire myself, but ultimately I find it very hard to actually be comforted by this sort of thing. I'm sort of hesitant to write about why, both because I hope other people can find comfort in this without me Eeyoring all over the comments, and in part because, like everything I write about this topic, it is going to make me sound very weird in some ways, but it is important to how I, and maybe some other people, relate to this weird big thing when we look at it too long.

1. Adding these details makes everything feel even less real. There's of course a little rational basis for this:

https://www.readthesequences.com/Burdensome-Details

But it feels even harder to grasp than I think even this suggests. There was a post on here a little while ago I can't quite manage to find, that basically talked about how the genre of thing AGI risk was looked really suspicious and rapturey in a way that should maybe raise alarm bells. I'm not so sure about this, I think in a way this genre of prediction is something of an attractor to many people and suspicious because of it, but even that isn't the end of the world (no pun intended). I would kind of rather that occasionally some group of smart, dedicated people throw their life work at some false alarm rapture (at least if the dynamics of the relevant groups they are in are otherwise healthy and reasonable) than, well, this:

https://astralcodexten.substack.com/p/heuristics-that-almost-always-work

But this genre does make it harder for people like me to viscerally feel anything about the event at all. The most I can usually manage is seeing it as like, some sort of big digital asteroid headed for us, not as an unborn god.

2. When I actually do take AGI seriously emotionally, my reactions are really weird. I'm probably not alone in this, but over the last few months, starting with Yudkowsky's doom post, and escalating with all of the capabilities developments that got released, this risk did suddenly feel emotionally real to me in a way it never did before, and mostly this was just emotionally devastating. Probably in part because it is much easier for me to feel the digital asteroid than unborn god thing. It has led me to realize that I have some really weird feelings about the whole thing.

One interesting thing I have sometimes seen people point out is that, selfishly, you really ought to hope that AGI comes sooner, because if it comes during your lifetime, you have a chance of living indefinitely, and if it kills you, well, you would have died only a little while afterwards anyway. Therefore, if you really hope that it does come later, because that increases the odds that it goes well, it means you are a really genuinely altruistic person. I do hope that AGI comes later rather than sooner, but on reflection, this is not an altruistic feeling, but just a really really confusing one.

I would prefer that AGI arrive one day after my natural lifespan ends than one day before. Maybe I am just not enough of a techno-optimist transhumanist type in my heart of hearts, but it just seems so horrible to me that I might be there when everything and everyone I love ends, even if there's a very good chance that won't be the result and things will instead be great. I think this might be sort of similar to how parents are able to stand the idea that their children will one day die. Maybe it is easier to cope with when they think to themselves "but I'll be dead by then anyway", even if this comfort ultimately isn't grounded in something in itself very different - I think if a parent learned they would die fifty years later, they would suddenly be much more scared of their child's death even if it were to happen at the same time.

I am also much more sad at the idea that something like factory farming could be ended by everything ending, rather than by us actually fixing anything, than I am made hopeful by the idea that the future could be great. I really don't understand any of these feelings, none of them are what I would have predicted, but they really suck, both personally and for my priorities, and I'm perversely grateful that, mostly by purposely reading less and less about AGI recently, I am managing to make it feel more unreal to myself again.

Maybe the takeaway from all of this is that I am personally a real bummer, I'm struggling with some other mental health problems right now anyway that maybe distort how I'm able to relate to this issue. Also I probably maybe just shouldn't work on AI safety. But anyway, this is some of the weird ways my reactions to this issue have gone in practice.

I think I mainly agree with the other comments (from Devin Kalish and Miranda Zhang), but on net I'm still glad that this post exists. Many thanks for writing it :)

Specifically, I think positive/x-hope (as opposed to negative/x-risk) styled framings can be valuable. Because:

1. There's some literature in behavioral psychology and in risk studies which says, roughly, that a significant fraction of people will kind of shut down and ignore the message when told about a threat. See, e.g., normalcy bias and ostrich effect.
2. Mental contrasting has been shown to work (and the study replicates, as far as I can tell). Essentially, the research behind mental contrasting tells us that people tend to be strongly motivated by working toward a desired future.^[1]
3. Anecdotally, I've found that some people do just resonate strongly with "inspirational visions" and the idea of helping build something really cool.

1. ^{^}

   Technically, mental constrasting only works if the perceived chance of success is high. I'm not sure what percentage "high" in this context maps to, but there is an issue here for folks who have a non-low P(doom) via misaligned AI... To counter this, there is the option of resorting to the Dark Arts, self-deception in particular, but that discussion is beyond the scope of this comment. See the Dark Arts tag on LessWrong if interested.