Messy personal stuff that affected my cause prioritization (or: how I started to care about AI safety)

Thanks for this post Julia! I really related to some parts of it, while other parts were very different from my experience. I'll take this opportunity to share a draft I wrote sometime last year, since I think it's in a similar spirit:

I used to be pretty uncomfortable with, and even mad about, the prominence of AI safety in EA. I always saw the logic – upon reading the sequences circa 2012, I quickly agreed that creating superintelligent entities not perfectly aligned with human values could go really, really badly, so of course AI safety was important in that sense – but did it really have to be such a central part of the EA movement, which (I felt) could otherwise have much wider acceptance and thus save more children from malaria? Of course, it would be worth allowing some deaths now to prevent a misaligned AI from killing everyone, so even then I didn’t object exactly, but I was internally upset about the perception of my movement and about the dead kids. 

I don’t feel this way anymore. What changed?

1. [people aren’t gonna like EA anyways – I’ve gotten more cynical and no longer think that AI was necessarily their true objection]
2. [AI safety more concrete now – the sequences were extremely insistent but without much in the way of actual asks, which is an unsettling combo all by itself. Move to Berkeley? Devote your life to blogging about ethics? Spend $100k on cryo? On some level those all seemed like the best available ways to prove yourself a True Believer! I was willing to lowercase-b believe, but wary of being a capital-B Believer, which in the absence of actual work to do is the only way to signal that you understand the Most Important Thing In The World]
3. [practice thinking about the general case, longtermism]

Unfortunately I no longer remember exactly what I was thinking with #3, though I could guess. #1 and #2 still make sense to me and I could try to expand on them if they're not clear to others. 

Thinking about it now, I might add something like:

4. [better internalization of the fact that EA isn't the only way to do good lol – people who care about global health and wouldn't care about AI are doing good work in global health as we speak]

Thank you for sharing!

My concern about people and animals having net-negative lives has been related to what’s happening with my own depression. My concern is a lot stronger when I’m doing worse personally.

I share the experience that my concern is stronger when I am in a worse mood but I am not sure I share your conclusion.

My concern comes from an intuitive judgement when I am in a bad mood. When I am in a good mood it requires cognitive effort to remember how badly off many other people and animals are.

I don't want to deprioritise the worst off in favour of creating many happy lives in the future just because I have a very privileged life and "forget" how badly off others are.

For what it's worth, my experience hasn't matched this. I started becoming concerned about the prevalence of net-negative lives during a particularly happy period of my own life, and have noticed very little correlation between the strength of this concern and the quality of my life over time. There are definitely some acute periods where, if I'm especially happy or especially struggling, I have more or less of a system-1 endorsement of this view. But it's pretty hard to say how much of that is a biased extrapolation, versus just a change in the size of my empathy gap from others' suffering.

I think there's something quite interesting here...I feel like one of the main things I see in the post is sort of the opposite of the intended message.

(I realise this is an old post now but I've only just read it and - full disclosure - I've ended up reading it now because I think my skepticism about AI risk arguments is higher than it's been for a long time and so I'm definitely coming at it from that point of view).

If I may paraphrase a bit flippantly, I think that one of the messages is sort of supposed to be: 'just because the early AI risk crowd were very different for me and kind of irritating(!), it doesn't mean that they were wrong' and so 'sometimes you need to pay attention to messages coming from outside of your subculture'.

But actually what happens in the narrative is that you only start caring about AI risk when an old friend who 'felt like one of your own' - and who was "worried" - manages to make you "feel viscerally" about it. So it wasn't that, without direct intervention from either 'tribe', you actually sat down with the arguments/data and understood things logically. Nor was it that you, say, found a set of AI/technology/risk experts to defer to. It was that someone with whom you had more of an affinity made you feel like we should care more and take it seriously. This sounds sort of like the opposite of the intended message, does it not?. i.e. it sounds like more attention was paid to an emotional appeal from an old friend than to whatever arguments were available at the time.

Thanks for sharing, Julia. I think this sort of post is valuable for helping individuals make better cause prioritization decisions. A related post is Claire Zabel's How we can make it easier to change your mind about cause areas.

Providing these insights can also help us understand why others might not be receptive to working on EA causes, which can be relevant for outreach work.

(Erin commented "people aren’t gonna like EA anyways – I’ve gotten more cynical", but I'm optimistic that an EA community that better understands stories like yours could do things differently to make people more receptive to caring about certain causes on the margin.)

This is a great post and my feelings have been almost identical over the years.

Thank you for sharing! I also appreciate you being so candid with your depression - I've found that many EAs are reticent bring up personal/health issues given that the movement mostly impacts us on a quasi-professional basis.

Also see Wikipedia on affective forecasting and projection bias. (h/t the book Stumbling on Happiness; note that I'm not sure whether this field of study was hit by the replication crisis)