AI Safety is bottlenecked by context, not talent (BC #19)

This post was inspired by my friend Jen’s quick take. I’m repeating some of her points, but I wanted to add my 2 cents. I’m also going to toot my own horn and refer you to a previous post of mine about the importance of context and how to build it. I recommend reading that first if you haven’t already.

The Bottleneck Paradox

We have been hearing a lot about the talent bottleneck in AI Safety. In some ways this is true, but in others it’s misleading and leaves many people disillusioned and doubting whether they have anything to contribute to AI Safety.

Meanwhile, the field is becoming harder to penetrate. Some reasons for this are:

• Organisations are better funded than they were a few years ago, which means they are less keen on working with volunteers and just want to hire full-time staff.
• Many of the “oldtimers” who could give the best career advice are now quite senior and oversee a larger number of people, which significantly raises the bar for talking to newcomers.
• The field has grown a lot, and many roles require more specialised AIS knowledge. There is more to read, and if you are new, it’s not easy to figure out which sub-area you should focus on.

Hell, I  run into people with pretty impressive backgrounds, but they can't even get into a BlueDot course.

So what’s going on?

Organisations are desperate to hire, but mainly for roles that require both many years of work experience and the high levels of context, which you build by having been around for years in the space. Here is a comment from Tom, who is a senior advisor at Probably Good.

A slight qualifier here is that getting to the level of context required for some jobs - especially senior ones that experienced professionals might be applying to - can take (sometimes much) longer, so it's important to have realistic expectations there. For instance, if you want to work in AI safety, and have a background (e.g. quantitative finance, venture capital) that could give you great skills to be a grantmaker, you'll likely still need to know more than just the high-level concepts and the landscape of organisations working on it; you might need to know the strengths and weaknesses of different theories of change, and have a sense of the wider funding landscape.

The people who can fill those roles are few and far between, and usually have already been working in AI Safety for years. If they become available, the headhunters are on their tails.

Some roles don’t need as much context, but:

• If there are more openings (e.g. operations^[1]), they get a large number of skilled applicants

• If they require a more niche background (e.g. marketing), then there aren’t as many opportunities available

Conclusion

Instead of saying AIS is bottlenecked by talent, I would like us to offer a more nuanced explanation, which looks something like this:

Some AI Safety roles demand a specific skill set, deep context, and years of work experience, which makes them hard to fill even when fill even when funding and talent aren’t constraints. Other roles are much more competitive, drawing many applicants since they don't require candidates to already have been around in AI Safety for a long time.

See also: Good Senior People Exist and We’re Wasting Them

1. ^{^}

    Though it’s worth noting that orgs seem to want high-context people for senior ops roles too, whether or not this is actually needed.