Many of us in this community are in the shocking position of thinking there’s a real chance of humanity being wiped out over the next decade or two. Most of the time, we discuss that in rational terms. We talk about probabilities, and threat models, and interventions. We don’t talk as much about the emotions we have about how radically our world might change and about the possibility of it ending entirely. 

There are lots of reasons for not talking about those feelings. For starters, it’s often hard to know how we even do feel about it. There isn’t a straightforward societal script for how to feel about such radical world changes. People each have to figure it out for themselves, and feel very different ways. No one wants to sound extreme or crazy by talking about feeling very strongly about it. But they don’t want to sound callous either. And opening up about your feelings and being met without understanding and similarity feels alienating, particularly when it’s about something so important. But the biggest reason I don’t talk about it is horror. I don’t want to think about it, and I don’t want to upset others. 

Until recently, I hadn’t thought about this lack of discussion as harmful. But someone recently highlighted to me how unnatural it can feel. It’s an unimaginably enormous thing to to come to terms with, and that’s all the harder if you feel alone in your emotional processing. Our community is also unusually keen to ensure they’re doing right by the world, so I think we’re unusually likely to worry about having the ‘wrong’ emotional reaction to things. I’m hoping that by describing a range of reactions I’ve come across, including my own, people can get a better sense of there being a significant range of both type and strength of emotions. Hopefully whatever your reaction to the world is, you can feel less alone in this struggle. 

A range of feelings

Here are some of the ways I’ve experienced people responding to fast AI timelines and uncertainty of survival: 

• For one of my friends, it only really clicked when they talked to someone in a similar family situation about risk from AI. Seeing someone they related strongly to have a strong reaction brought things home to them in a way that talking to others hadn’t.
• For another person, an increase in the speed of AI progress compared to their expectations caused a couple of months of low mood and depression, as they came to terms with a different, more dangerous world
• One of my friends used the analogy of walking over an incredibly high glass bridge. Much of the time, he’s looking ahead and everything is normal. But sometimes he look down and the vertigo is overwhelming. He has a naturally high hedonic set point, so it doesn’t feel harrowing so much as incredibly disorienting and stressful.
• For another, the feeling is more clearly negative and occasionally brings bouts of tears when he really needs to think through what the future might look like, and the implications.
• For one person I’ve discussed this with, disorientation about the future makes it hard to make ‘normal’ decisions about the future, because it’s no longer possible to picture what the options look like. In the past it’s been clear what it looked like to be a parent through the different stages of your child’s life - no longer.
• A friend of mine with particularly short timelines and high expectation of doom keeps herself calm and forging ahead by intentionally staying in the frame of ‘we will win’.
• I’ve heard people describe feelings of isolation about this. Sometimes that’s isolation of being surrounded by people who don’t take seriously the possibility of huge upheaval this decade, and so facing something disconcerting alone. Sometimes it’s isolation from feeling sympathetic to both people taking these ideas seriously and those who don’t, and therefore feeling like others have a ‘tribe’ and they don’t.
• Neither are all the feelings persistent, or ones that feel at all endorsed. Obviously humans have all sorts of emotional reactions, particularly to things that are so foreign, like dark humour at the ridiculous situation we find ourselves in. 

This tiktok really resonated on our team’s slack.

How I feel 

One thing to say is that I’m pretty cautious about sitting too long with my feelings about this, or going too deep into them. I’m pretty worried about doing so having a lasting effect on my ability to enjoy things and be productive. I don’t know how realistic vs paranoid that is. 

I find it pretty difficult to look at the chance of the world not going well. I do think it’s important to try to build a concrete picture of what the future might look like, what specific dangers we might face and how we might avoid those. I’m particularly grateful to the AI 2027 team for producing content which nails that. But reading it left me in tears. 

I resonate with the analogy of walking over a glass bridge. I feel the weird juxtaposition between being often in the mode of ignoring future changes and then periodically actually recognising them. And vertigo feels like a good analogy to the disorientation of looking at my current guess of the future. I like predictability, and knowing what the future looks like and how to plan for it. The idea of a century’s changes in a year sounds terrifying. 

But that pales in comparison to the idea that my son might not reach adulthood. 

Recently, the spouse of a friend of mine was diagnosed with a condition that had a 1% chance of being fatal. That really struck my emotions hard. For pretty much all my adult life, my biggest fear has been my husband dying. I couldn’t fathom what my friend and his family were going through. But I’m not at all confident enough to say that humanity is at least 99% likely to survive transformative AI. 

When thinking about the possibility of my husband dying, it actually slightly takes the edge off the sadness if it’s a situation in which I die too. But that’s not at all true for my son. He will be such a wonderful man, I can’t bear the idea he might not get there. When I get too much sucked into thinking about that, I remember what a happy child he is, and that it’s so good that he’s gotten to be alive. One time when I was struggling with this, a friend of mine pointed out that in expectation AI will extend my son’s life, because of how much it could extend it if things go well. I find that surprisingly reassuring to remember.

Something I haven’t looked at approximately at all is the chance of S-risks caused by transformative AI. Considering situations where my son suffers ongoingly is absolutely intolerable.

Different people are different 

Cognitively, what I think is bad about risk from AI is the potential of it wiping out or significantly and perpetually curtailing value across the universe for trillions of years. But what my feelings focus on is very often my son. Others might be sad about others in their family. They might be terrified of death, or feel anger and betrayal that humanity would cause its own destruction by failing to prioritise the wellbeing of humanity over short term profit. Some people likely actually feel at a deep level the weight of the colossal loss of value echoing down the centuries. 

The things that allow people relief when thinking about the future will likewise differ. For someone whose child has a cancer diagnosis, the possibility of shocking medical breakthroughs happening soon could easily eclipse many other emotions.

Some people might not feel strongly about these risks at all - they may simply think about them on an intellectual level. That doesn’t seem surprising either: it’s hard for our brains to wrap themselves around things they haven’t experienced before. 

Our background hedonic state also makes a huge difference to how we experience things - both in terms of strength, and valence. Sometimes I feel a calm peace that the world will work itself out, without having any evidence that it will. I’m just naturally inclined to feel that people will act kindly, and (at least in some moods) that endings are happy.

Some people might want to further investigate their feelings about the future. It might make things feel more real and give you a better sense of what you might do to make things go better. Or getting in touch with your feelings about it might make you more motivated to work on improving the expected future or more able to motivate others to work on it. It might help you feel more like you’re orienting appropriately and authentically to the world. 

Personally, I want to keep trying to understand ways the future might go and their probabilities, ideally in concretely graspable terms. That sometimes bleeds over into me having the sense that I ought to look the future more squarely in the face, feelings and all. There are many things I could come to terms with if I internalised and processed them. I’m pretty worried that my son’s death isn’t one of them, and that if I stare too hard at my expectation of the future, I’ll break. 

I’m guessing that some people are like me in finding it hard to properly remember that your duties to the world are about helping improve it, not feeling its sadness. I find it useful to have others around me remind me of my endorsed view, and to feel permission to look away from rather than towards the sadness. For those who are like me, I hope my writing this can help you feel the same. If you’ll go blind staring at the sun, use instruments to learn its brightness, don’t try to feel the brightness yourself. 

This post was inspired by a conversation with Josh Rosenberg. He, Julia Wise and Jess Binksmith commented on and improved it.