A response to Gergő and Aaron’s recent essays on context and worldview.
My Perspective on the Disconnect
As someone who’s been working to transition to AI safety for the past 6 months, I’m quite familiar with the context & worldview gaps that Gergő and Aaron both wrote about recently. If you’re assuming I intend to argue against their importance you’re mistaken – the gaps are real. They are two sides of the same disconnect, and this disconnect is a far bigger challenge for the AI safety community than it may appear to be.
When I first began exploring opportunities, I wasn’t aware how large the disconnect was. I saw all the “we need talent” messaging, and, with a very successful background helping employers find talent, I assumed it would be easy to engage and join an org looking to scale.
Rejection does a funny thing to someone like me. I’m competitive, I’m a problem-solver, and I don’t quit on things that I care about. And although I came in with a casual understanding, I quickly became consumed by the AI safety mission and committed to the cause. After not getting any interviews and even being rejected for the BlueDot AGI course, I decided to sunset my business and go all in on learning about AI safety. I stopped sending applications into the void and spent my time reading, writing, and thinking about AI risks as if it were a full-time job.
After almost 3 months, I reapplied to BlueDot with a much stronger foundation of understanding. This time I was accepted, and since my initial rejection led me to assume that everyone in the course would be very knowledgeable, I looked forward to connecting with others that would match the conviction and concern I had developed over the past few months.
I couldn’t have been more wrong.
The people in my group were all sharp and seemed highly capable, but they weren’t anywhere near as concerned as I was. While I came in having already deeply engaged with many of the readings, it seemed as though they were being exposed to some of these ideas for the first time.
I had found myself on the other side of the worldview gap.
Unit 3, the one on pathways to harm, was where the disconnect hit me the hardest, and I walked away from that discussion feeling more than a little disoriented. Since I’m the type that processes through writing, that’s exactly what I did. I won’t share the full essay I wrote immediately after the session, but I will share some of the comments from my peers that led to my disorientation:
“I don’t think authoritarianism is a pressing concern.”
“These scenarios are depressing. I’d rather be optimistic.”
“I can see how things could go bad, but I don’t think that the people in charge will let that happen.”
“The technology isn’t good enough for that to actually be possible, and even if it was I don’t think that would qualify as an existential risk.”
“I don’t believe we should worry about that, but I didn’t do all the readings so maybe I’m missing something.”
Obviously my experience isn’t reflective of everyone who participates, but it left me craving a level of engagement and understanding that I simply wasn’t able to find among the peers I was exposed to. Since then, things have changed a bit on my end as I’ve shifted my focus from only learning to now intentionally seeking out that kind of engagement. I've connected with SPAR fellows working on projects where my expertise is relevant and have been sharing some of my writing on Substack, both of which have been far more satisfying than filling out job applications. But I’m not posting this here to give an update on my journey.
I’m posting it because the disconnect from these gaps is far more consequential for those already in AI safety than it is for the people trying to work in AI safety.
The Larger Consequences
These same gaps that keep qualified people out of AI safety orgs are what keep AI safety from getting support from the rest of the world. The number of people who think about AI risks regularly is relatively small, and the vast majority are already working in the field or actively trying to enter it. That’s not to say members of the general public don’t have opinions on AI risks, they certainly do, but that hasn’t translated to sustained attention and engagement, and when it has, their focus centers on matters far removed from the priorities of the AI safety community.
AI safety needs to find new ways to engage with the public and build support for the cause.
This might be hard to recognize for many within AI safety orgs that are focused on doing good work and don’t feel the need to adjust their approach. As someone coming from the business world, I can’t help but look at the circumstances through a different perspective. You could build the best product and still see your business fail if you don’t know how to sell it. It doesn’t matter how good the research may be or how accurate the predictions are if the work doesn’t translate to meaningful influence on the broader power landscape. Through that lens, AI safety is failing.
We are currently on the Plan D pathway, and we’re ahead of where the top AI safety forecasters predicted we’d be at this point. RSI is either here or around the corner, and we could be looking at an intelligence explosion very soon. And while adoption may be slower than anticipated, that does little to curtail the risks we may be facing on the path ahead.
The frontier labs aren’t listening. The government undermines AI safety and calls AI risks a hoax. Media coverage centers on scandals and controversies, doing little to further the cause. The public is largely indifferent and focused on their individual lives, and those who aren’t are far more concerned about job displacement or the idea that “data centers waste water” than the existential risks. Waiting for another warning shot and hoping it draws people into the cause is not a viable strategy. AI safety needs to go on the offensive, and it needs to start sooner rather than later.
To be clear, I’m not saying the community needs to change what it focuses on. Keep doing the work. But if you want that work to be as meaningful as possible, new approaches to engaging with the public should be considered, and individuals that can drive those efforts should be welcomed. The field has spent considerable effort trying to get people outside of AI safety to understand its worldview, when what it should be doing is figuring out how to frame and deliver that worldview so it may actually influence the direction we are headed.
I’ve spent the past handful of weeks immersed in ideas related to that, and I have some strategies in mind that I believe could position AI safety to more successfully raise awareness, build support, and drive meaningful change. I’ll save those for another post, but I’d love to connect with anyone else who recognizes this problem and wants to get involved or discuss what I’ve been working on.
You can message me here, on Substack, or send me an email.
It seems reasonable to me that courses designed to introduce people to AI safety would attract participants encountering some of these ideas for the first time. That's what the courses are there for, right?
Your coursemates being less alarmed also doesn't mean they were less informed. Even people working at the frontier labs seem to disagree about how concerned we should be.
But the question I'm itching to ask is - given that you seem to have acquired the kind of context that the earlier post identified as a missing ingredient among candidates, and given your extraordinary efforts over a prolonged period ( that is not sarcasm; you really do seem to be living and breathing this topic), have you managed to secure a role in AI safety yet?
I definitely see your perspective! Getting rejected from the course the first time I applied (back in June) largely influenced my assumption that I needed to strengthen my understanding if I wanted to get in. For all I know, I could have applied a week later and been accepted back then, and had I done so I'm sure the experience would have been very different.
On coursemates being less alarmed, I agree. A lower level of concern doesn't necessarily mean someone is less informed. I'm sure there are plenty of people that know far more than I do and simply disagree with my perspectives. That said, in the discussions I participated in, many of my peers shared that this was their first time encountering some of the arguments or even that they hadn't done all of the readings. That's not a knock on them - many are still working full-time roles, and there's a lot of literature to engage with in order to develop a solid foundation of understanding. Not everyone can stop what they are doing and commit full-time to learning about a field they want to enter, and, to me, that's largely why these context/worldview gaps exist in the first place.
And no, I haven't secured a role, but I also haven't applied to any since September. My focus has shifted toward developing a concept for a new organization aimed at addressing some of the gaps I see in the field. The types of roles I was initially targeting are fairly senior level and relatively few and far between, and I think there are plenty of meaningful areas of impact that a new org could pursue.