TL;DR: AI safety spends a lot of effort bringing people into the field, but I’m less sure we pay enough attention to what happens afterward: whether people find roles and managers that work for them, build lives around the work that they can sustain, and stay connected to people outside the ecosystem. Burnout and difficulty switching off seem like real problems, and I think some of the broader social and professional concerns are worth probing too. I’m running a SPAR project to investigate one part I think is especially underexplored: how deeper involvement in AI safety affects relationships with family, friends, partners, and former colleagues, and what those people can tell us about how the field looks from outside its own norms.
Cross-posted from Clarity is Courage.
A lot of AI safety field-building is, understandably, oriented toward expansion. There are university groups, fellowships, retreats/workshops, career-transition grants, and now incubators popping up across the ecosystem. Funders are putting serious money behind new organizations and talent programs. We spend a lot of time thinking about who should enter the field, where the talent bottlenecks are, and how to get capable people working on important problems. I’m excited about most of this; I’ve spent a fair amount of my own time thinking about talent pipelines, especially early-career talent through university organizing.
I’ve started wondering, though, whether we think as carefully about what comes after someone enters the ecosystem. Through the Generator Residency, I’ve spent this summer at Constellation, where people from many of the main AI safety organizations work. I’ve seen a fair amount of burnout and heard similar concerns from other residents and people across the field; recent survey data makes me more confident that at least the burnout and difficulty-switching-off pattern is not Generator-specific and shows up more widely across the field. Some of the other patterns below are more speculative, but I think they’re worth probing.
A lot of field-building has a fairly legible theory of change: find promising people, give them context and training, connect them to useful people and opportunities, and hopefully some of them go on to do important work.
The early stages are also relatively easy to measure. We can count applications, attendance, fellowship completions, career changes, placements, organizations launched, and money raised. The growing number of incubators and talent programs across AI safety is, among other things, a bet that there are more useful people and projects we should be bringing into the field.
What happens later is harder to see. Someone can look like a success on the usual metrics and still end up in a role that fits them poorly, burn out quickly, discover that they hate the city they moved to, or slowly realize that the life they built around the work is not one they want to keep living. Sometimes the reason is fairly mundane: bad management, a move that doesn’t work for their partner, or old relationships becoming much trickier to maintain.
Those things matter for the person, but also for whether the original talent investment amounted to much. If we spend a great deal of effort getting someone into AI safety and they are gone six months later, or spend those six months badly matched, badly managed, and unhappy, the conversion statistic has hidden a lot.
This is one reason I worry that we can end up underweighting maintenance. The benefits are slower, less attributable, and much harder to put in an impact report. It is easier to point to another cohort of thirty people than to someone who remained effective for five years because they found a role that fit, had a decent manager, and built a life around the work that they could sustain.
For some people, moving into AI safety is a fairly normal career change. For others, it is still a pretty strange thing to do with your life.
I know people who have effectively treated organizing their university AI safety group as a full-time job while doing the bare minimum needed to stay enrolled. Others drop out, leave stable careers, move across the country, or reorganize large parts of their lives around a problem that many people close to them barely think about.
The social transition can be substantial too. In the Bay, it is easy to work with AI safety people, live with AI safety people, eat lunch with AI safety people, go to an AI safety event that evening, and spend the weekend with friends you met through the field. After a while, a large share of both your professional and social world can come from the same ecosystem.
There is plenty I like about this; I benefit from it myself. Being around people who understand why you care about something makes collaboration easier and can make an otherwise unusual career feel much less lonely. A lot of valuable work probably happens because the people involved are physically and socially close to one another.
It can also become hard to turn the subject off. One Generator resident recently put it more bluntly: “we should remember how fucking insane all of this sounds to everyone else.” People here casually talk about takeoff speeds, whether humanity survives the century, digital minds, and space governance. To some people, RSI is an outdoor clothing company. Someone else remarked that the Riemann hypothesis sounds like a Dustin Hoffman movie.
A close colleague recently told me he was bothered by how much he had picked up the gestures and speaking cadence of people around him in AI safety, enough that somebody else could identify who he had been influenced by. People imitate their friends, coworkers, and people they look up to all the time, but this makes me think more about how those things had stopped feeling noticeable from the inside.
I can relate to that myself. Spending most of the summer at Constellation has changed who I talk to, what I talk about, what career choices seem reasonable to me, and probably a bunch of things I have not noticed yet. I’m mostly glad it has. I also think there is value in retaining some awareness that my reference point has moved.
For some people, that immersion also makes it much harder to switch off.
I saw a fair amount of burnout during Generator. Some people felt isolated; others had stretches where the rest of their life was clearly not going very well.
Julian Hazell published a post this week pointing to recent survey data that makes me somewhat more confident this is not just something I happened to notice in one residency. In a survey of around 80 people working on existential risk, AI safety, animal welfare, and other high-impact causes, 68% reported difficulty switching off from work, 59% burnout, and 45% had considered cutting back or leaving. The sample is small, self-selected, and broader than AI safety specifically, but the difficulty-switching-off result especially caught my attention.
A Generator resident came into the summer already well connected in AI safety and fairly sure they were going to make a major life change to work in the field. They burned out quickly and ended up moving back toward a more conventional path, while also stepping back from some of the AI safety work they had already been doing. What interested me was that workload alone didn’t seem to explain what happened. They already had trouble separating work from personal life, and being in the middle of the Bay AI safety scene made that harder. You leave your desk and people are talking about AI safety over lunch. You go home and your housemates work on AI. You go to an event and somehow you are still talking about rationalism.
A senior program manager who has worked closely with people across the field gave me another hypothesis: burnout here can be high even when people aren’t working the longest hours because the moral stakes make it harder to disengage. If you think your work is connected to reducing existential risk, there is always another reason to keep going, and stepping away can feel harder to justify to yourself.
The age distribution probably matters too. A lot of people enter AI safety young, sometimes before they have had much time to figure out what kind of work they are good at, how hard they can sustainably push, how to set boundaries, or how much of their identity they want tied up in a career. Then we hand them one of the most morally loaded career questions possible. Spencer Greenberg’s discussion of the survey points in a similar direction: the more common pattern was chronic strain around rest, motivation, and boundaries rather than clinical anxiety or depression.
Career decisions in this field can start to feel like an optimization problem: personal fit, comparative advantage, counterfactual impact, status, compensation, geography, what jobs happen to exist, what your peers think is valuable. I catch myself thinking in these terms too, and most of them are useful. But taken together, they can turn a career decision into something that feels like a referendum on whether you’re doing enough with your life.
Not everything I’m worried about belongs under community health. Several Generator residents were surprised by fairly basic collaboration problems with more senior people they worked closely with. Expectations weren’t always clear, communication could be poor, and balls sometimes got dropped when they weren’t urgent. What caught my attention was that some people can be remarkably sharp and effective when something is on fire, while being much less consistent on ordinary follow-through.
I can imagine a field built around very high stakes strongly rewarding the ability to mobilize around urgent problems, while putting less weight on the boring habits that make someone consistently good to work with. If everything is supposedly on fire, ordinary management work like clear expectations, reliable follow-through, useful feedback can feel somewhat beneath the urgency of the field.
If a junior person spends a week confused about what their manager wants, that is both unpleasant and wasteful. If enough strong people have these experiences and update negatively on how well the field is run, the cost gets larger still. For an effectiveness-oriented community, this should not be hard to care about.
Julian makes a closely related point about mental health: AI safety has a high willingness to pay for interventions that bring new people into the field, so in principle it should also value interventions that keep people already here effective. I think that logic extends beyond mental health to management, career fit, social support, and other things that are much harder to quantify.
I’ve been thinking about this especially in the context of mid-career transitions.
I like the basic idea a lot. Someone who has spent ten years becoming excellent at law, cybersecurity, journalism, policy, management, operations, or another field has skills and judgment that we are not going to reproduce with an eight-week fellowship.
Mid-career people also have more to compare us to. Someone coming from a good security team, newsroom, law firm, government office, or consultancy has years of experience with what competent collaboration looks like. They will notice weak management and unreliable follow-through. They may also notice parts of AI safety culture that feel inward-looking or socially odd in ways that someone who entered the field at nineteen doesn’t quite register.
They also have more to give up. A mid-career person may already have a career, status, close friends, a spouse, children, a professional network, and a city they like. It is not enough for them to conclude that AI risk matters; there needs to be a sensible place for them to contribute and a transition that works with the rest of their life.
This is where I sometimes feel uneasy about how career-transition programs are discussed. Context, introductions, retreats, and grants can all help. But eventually there still needs to be a role that uses what someone spent ten years getting good at, a plausible path into it, and an ecosystem that looks competent enough that leaving a successful career for it doesn’t feel like joining something held together by vibes.
The social side matters here too. If someone comes from a fairly conventional professional environment and encounters a community where social networks overlap heavily, people speak in the same compressed vocabulary, copy each other’s mannerisms, and work and social life blur together, they may find that exciting. They may also find parts of it a tad cult-ish.
The more effort we put into mid-career transitions, the more these details matter.
One part of this seems small enough to investigate directly: how people working in AI safety relate to family, partners, old friends, and former colleagues. These people are not a representative sample of the public, and interviewing twenty or so of them would tell us very little about public opinion generally. What makes them interesting is that they know us, and often knew us before AI safety.
A parent can think AI risk is real and still think their child is crazy for dropping out. An old friend can take the work seriously while thinking the person doing it has become more intense and harder to talk to. A former colleague can respect the problem and still think the organizations around it look amateurish. I think we blur these things when we talk about whether someone is safety pilled. Believing the underlying problem is real, trusting the field, being comfortable with the community, and thinking one person’s life choices make sense are all different judgments.
People who knew someone before they entered the field also have a reference point that newer friends and coworkers don’t. They won’t always be right, but they may notice changes we miss.
There may be an epistemic benefit here too. If most of the people you work and socialize with share a bundle of assumptions, maintaining close relationships with people who don’t share them gives you another place to check whether your reasoning, language, or behavior still makes sense outside the immediate environment. That is part of why I’m interested in these relationships both as sources of support and as sources of calibration.
I’m mentoring a project through SPAR this fall to investigate this part more.
The plan is to interview people working in AI safety about how deeper involvement in the field has affected relationships with family, partners, friends, and former colleagues. Where possible, we’ll also talk directly to some of those people rather than asking AI safety workers to guess what everyone else thinks.
I want the interviews to stay close to specific experiences: explaining a career change, moving to the Bay, a conversation that went badly, an old friendship becoming harder to maintain, or realizing that nearly everyone you now spend time with works on the same set of problems.
The main output will probably be a practical guide for people navigating these situations, alongside a short synthesis of what the interviews tell us about how involvement in AI safety affects relationships and how the field looks to people close to those working in it.
I’m not confident that the problem is as large as I currently suspect. Generator and Constellation are both intense environments, and maybe other high-pressure fields produce many of the same patterns. If the first ten interviews suggest that people’s relationships are mostly fine and that being surrounded by AI safety people is overwhelmingly helpful, I would update.
I also like this as a Generalist Megastream project because it starts with concrete work (e.g., designing interviews, recruiting participants, conducting them, organizing what we hear) but gets more judgment-heavy as it goes. A mentee eventually has to decide whether the patterns they’re seeing truly hold up, what other explanations might account for them, and whether the project’s direction should be pivoted.
I don’t think the takeaway is that AI safety needs to divert large amounts of money into community wellness. I also don’t think close-knit communities or periods of intense work are inherently bad. I’m more concerned that we have decent systems for bringing people into AI safety, but much weaker ones for noticing what happens afterward.
If someone burns out after six months, that matters for them and for the field. If an excellent mid-career person gets excited about AI safety but never finds a role that uses their skills, the transition has not done much good. If experienced outsiders encounter enough sloppy collaboration or social insularity that they conclude the ecosystem is not very credible, that affects the talent pool too.
There is also a more basic reason to care. We are deliberately encouraging people to make consequential decisions because we think this problem matters: leaving careers, moving cities, turning down more conventional paths, and devoting a large part of their lives to AI safety. I think that creates some responsibility to care about what happens after they say yes.
I don’t yet know how common these problems are, how much they matter relative to other bottlenecks, or which are especially specific to AI safety. But given how much effort goes into bringing people through the front door, spending some effort understanding what happens on the other side seems like worthwhile due diligence.
I’ll be digging into one part of this more closely. Join me! If interviewing people, making sense of messy qualitative evidence, and figuring out which parts of this story hold up sounds interesting to you, I’m looking for mentees through the SPAR Generalist Megastream. Apply here by August 18!
