New Answer
Ask Related Question
New Comment

2 Answers sorted by

Per Andy Jones over at LessWrong:

If you think you could write a substantial pull request for a major machine learning library, then major AI safety labs want to interview you today.

I work for Anthropic, an industrial AI research lab focussed on safety. We are bottlenecked on aligned engineering talent. Specifically engineering talent. While we'd always like more ops folk and more researchers, our safety work is limited by a shortage of great engineers.

I've spoken to several other AI safety research organisations who feel the same.

I'm not sure what you mean by "AI safety labs", but Redwood Research, Anthropic, and the OpenAI safety team have all hired self-taught ML engineers. DeepMind has a reputation for being more focused on credentials. Other AI labs don't do as much research that's clearly focused on AI takeover risk.

I'm currently at DeepMind and I'm not really sure where this reputation has come from. As far as I can tell DeepMind would be perfectly happy to hire self-taught ML engineers for the Research Engineer role (but probably not the Research Scientist role; my impression is that this is similar at other orgs). The interview process is focused on evaluating skills, not credentials.

DeepMind does get enough applicants that not everyone makes it to the interview stage, so it's possible that self-taught ML engineers are getting rejected before getting a chance to show they know ML. But presumably this is also a problem that Redwood / Anthropic / OpenAI have? Presumably there is some way that self-taught ML engineers are signaling that they are worth interviewing. (As a simple example, if I personally thought someone was worth interviewing, my recommendation would function as a signal for "worth interviewing", and in that situation DeepMind would interview them, and at that point I predict their success would depend primarily  on their skills and not their credentials.)

If there's some signal of "worth interviewing" that DeepMind is failing to pick up on, I'd love to know that; it's the sort of problem I'd expect DeepMind-the-company to want to fix.

5ElizabethBarnes1y
I think DM clearly restricts REs more than OpenAI (and I assume Anthropic). I know of REs at DM who have found it annoying/difficult to lead projects because of being REs, I know of someone without a PhD who left Brain (not DeepMind but still Google so prob more similar) partly because it was restrictive, and lead team at OAI/Anthropic, and I know of people without an undergrad degree who have been hired by OAI/Anthropic. At OpenAI I'm not aware of it being more difficult for people to lead projects etc because of being 'officially an RE'. I had bad experiences at DM that were ostensibly related to not having a PhD (but could also have been explained by lack of research ability). 
2Rohin Shah1y
I would love to hear actual details of these cases (presumably less publicly); at this level of granularity I can't tell to what extent this should change my mind (I can imagine worlds consistent with both your statements and mine).
3Buck1y
As I understand it, DeepMind doesn’t hire people without PhDs as research scientists, and places more restrictions on what research engineers can do than other places.
3Rohin Shah1y
Basically true (though technically [https://boards.greenhouse.io/deepmind/jobs/469515?utm_campaign=google_jobs_apply&utm_source=google_jobs_apply&utm_medium=organic] the requirement is "PhD in a technical field or equivalent practical experience") Doesn't seem true to me. Within safety I can name two research engineers who are currently leading research projects. DeepMind might be more explicit that in practice the people who lead research projects will tend to have PhDs. I think this pattern is just because usually people with PhDs are better at leading research projects than people without PhDs. I expect to see the same pattern at OpenAI and Anthropic. If I assigned people to roles based solely on (my evaluation of) capabilities / merit, I'd expect to reproduce that pattern.
4richard_ngo1y
"DeepMind allows REs to lead research projects" is consistent with "DeepMind restricts REs more than other places". E.g. OpenAI doesn't even officially distinguish RE from RS positions, whereas DeepMind has different ladders with different expectations for each. And I think the default expectations for REs and RSs are pretty different (although I agree that it's possible for REs to end up doing most of the same things as RSs).
2Rohin Shah1y
I continue to think that this is primarily a reflection of RSs having more experience than REs, and that a process with a single role and no RS / RE distinction would produce similar outcomes given the same people.
2Ansh R1y
As someone interested in applying for Research Engineering roles in the near future, what would be your criterion for determining whether someone self-taught is “worth-interviewing”? (Also a question for others who are familiar with hiring practices at various AI safety organizations).
3Rohin Shah1y
I don't know of some easy-to-describe bar, but as one anecdote, this post [http://amid.fish/reproducing-deep-rl] by Matthew Rahtz was easily enough to clear the "should interview" bar, and went most of the way to the "should hire" bar, when I was looking at applicants for the CHAI internship. It would also have been enough to clear the "should interview" bar at DeepMind. I also like this 80K podcast [https://80000hours.org/podcast/episodes/olsson-and-ziegler-ml-engineering-and-safety/] on the topic, and in general I might recommend looking at my FAQ [https://rohinshah.com/faq-career-advice-for-ai-alignment-researchers/] (though it doesn't cover this question particularly).