TL;DR: There is a grave discrepancy between hiring managers and candidates causing promising talent to abandon AIS. Risk-aversion to hire, talent pipelines inefficiencies and feedback negligence are among the causes of this trend, pushing us to a future where qualified talent is scarce. All this endangers AIS mission, needing to take urgent action to ensure we have the people capacity to mitigate catastrophic AI risks. This post shows a thorough analysis of the issue, while offering practical and simple to implement measures to alleviate it.
AI Safety is experiencing a weird discrepancy:
So, if organizations are lacking qualified professionals to support their scaling, and experienced professionals find themselves unable to join these companies, what the h*ll is going on?
Short answer: AIS pipelines and hiring processes are severely inefficient + Organizational risk-aversion to hiring is killing the field potential for expansion.
Long answer: A little more nuanced, bear with me.
Let´s analyse this situation through the lenses of a qualified applicant and a hiring specialist:
A qualified applicant (typically a mid-career generalist) might have convinced themselves to transitioning into AIS. They have followed the frontier AI news and disruptions such as the Hugging Face incident, plus heard the call of the field need of talent. They have spent the past months gaining context about the sector (BlueDot courses, AIS research papers, field needs, etc.), and applied to dozens of fellowships and open positions they believed to be qualified. Only to find rejection after rejection without useful feedback, and eventually abandoning their quest to contribute to the field after hundreds of hours invested, with the realization maybe this path was not for them.
A hiring specialist has recently joined a fast-growing AIS organization. They might have secured a big chunk of funding for the next 2 years and need to make a good use of them. The hiring specialist is in charge of running several hiring processes at the same time, being responsible of landing successful professionals into those positions. These roles are not urgent by the organization current necessities, but are critical for their growth objectives and to properly invest the secured funding and justify acquiring a next round. During the first weeks all roles receive a few thousands of applications. Even if the majority are discarded by the AI screening and the ATS, the hiring specialist still needs to review hundreds of them, communicate with candidates, evaluate work tests and run interviews. The stakes are high. Selecting the wrong candidate might undermine growth objectives and threaten a future round of funding.
The consequence of this setup is a poor hiring process that maintains the severe talent bottleneck AIS is currently enduring[1]. Plus, a quite negative experience "outsiders" professionals are receiving, one that we may heavily regret in the near future.
Consider the current moment AIS is experiencing regarding talent as an anomaly. Right now, qualified talent is abundant and organizations across the field can allow inefficiencies and poor experiences during their hiring processes. This talent abundance may be the result of the public consciousness about the dangers AI impose into society, and the most potentially aligned professionals are the ones who have made the first step.
This is very important so I am going to repeat it: The most potentially aligned professionals that can help AIS grow are already in the talent pool. The ones that answered the first call for the field talent need. The ones that are most committed to ensure AI is safely deployed into society. The ones that only needed awareness of a single security incident to make a move. They have been looking forward to contributing into the field for months.
And they are getting pretty disappointed.
Every rejection. Every application sent unanswered. Every email without meaningful feedback. All of them are new chances for this already available talent to give up out of frustration, and maybe to never come back.
Hiring managers and AIS organizations are obsessed to hire the 99th percentile of talent available. What they do not realize is that current talent within the 80th to 90th percentiles are the most aligned professionals to hire in the next few years, and we are heavily neglecting them. They are qualified professionals who may need some more context, a few guided improvements to be close to the best ones. But we are not giving them the chance.
And as the tail of the goose that laid golden eggs, we may end without the talent pool we are forgetting to look after.
Counterargument: "Sergio, this qualified talent abundance might not be an anomaly and we may continue benefiting from it". You might be right, although the risk is still worth considering. And again, the most determined professionals willing to help are already here. The ones that did not consider AIS worth contributing after the alarms rang are the ones you are expecting to come in the future more informed and qualified. Probably they will not be as aligned as the talent pool we currently have.
Consider the next 2 possible scenarios for the near future:
Both scenarios would be mitigated by taking decisive action in the present moment, implementing measures to help our current talent pool to move forward the pipelines. And this is a responsibility for the whole field, not only for programs or field building organizations. Everyone in contact with this potential talent has to play their part if we intend to succeed in the quest of mitigating severe and existential AI risks.
For more context regarding this issue I recommend reading this and this field studies conducted by Enisa (@Enisa Ismaili) and Uladzislau (@Uladzislau Linnik) of GenerAISts. They show enlightening data such as 39% of the transitioning senior professionals interviewed abandoning their attempt to enter a full time AIS job, after hundreds of hours spent and months of fruitless efforts.
Before discussing some practical measures to alleviate AIS talent bottlenecks, first we have to take a look at the elephant in the room.
There has been much discussion about new talent not signalling alignment as the main reason to be rejected on early-stages of hiring processes. If mostly true, we are confusing two different concepts of "alignment" that impair talent capabilities to improve and advance the pipeline.
When AIS hiring managers discuss alignment they usually refer to the "motivational and context awareness" part:
However, there is a second concept related to "values alignment" or simply "trust", that might play and even more important role during hiring decisions processes:
Here we are talking about the specific person fit into the organization, with their personality features and soft skills. These are harder to improve and much difficult to assess in short hiring processes. And this "trust signalling" is probably the one most hiring managers take as the cornerstone to recruit someone, event if they convey it as the "motivation and context awareness" one.
If AIS recruiters prefer to hire someone they met at an EAG event or referred by a trusted organization, versus an outsider qualified professional, it is because they are using those as proxies to ensure values alignment[2]. In other words: even when hiring managers publicly state they look for capable professionals who are motivated and context aware of AIS field, what they care most is neither of those features, but how much they can trust these professionals to behave inside their organization.[3]
Why are AIS founders and managers so risk-aversed to hire? What reason is behind caring so much about trust signalling?
We could argue the damage posed on a team moral after a bad hiring is notable. An organizational culture could be resented after the firing of a negative employee, whether they were indeed toxic for the team members or because other employees got attached to them. However, this is a hazard every for-profit company also faces, and their risk-aversion to wrongly hire seems much lower than in non-profits organizations.
Although we might need more data to firmly conclude this, I would argue a key factor behind this hiring risk-aversion has to do with funders perception. To demonstrate grantmakers the organization is capable of growing, that is properly expending the resources and deserves more funding, while not committing critical mistakes when incorporating people that could endanger all of it. And although this line of reasoning is understandable, this could be the main psychological cause behind AIS talent bottleneck, endangering the field mission.
I would like to ask funders and grantmakers which of these scenarios they prefer:
Most AIS companies are in the first scenario, especially regarding management and senior positions. The second scenario would probably allow unblocking an important stretch of the talent pipeline. But for that to happen, funders and grantmakers need to be clear on the signals they are emitting to AIS organizations in order to ensure future funding, regarding the rate of hiring failure/success they are willing to tolerate.
To sum up this section: Do we want AI Safety to play small and secure (risk-averse, without hiring mistakes) or do we want to solve one of the biggest threats humanity may be facing in centuries? This challenge requires speed, courage and swift action. But our hiring practices and talent pipelines are falling short... and we might regret it, unless the organizations leading this field change their course, soon.
Down below there are some suggestions that could alleviate the issues highlighted in this post. These measures are intended to be practical and relatively quick to apply, in order to ease the talent pipeline flow and improve both hiring managers and applicants experiences[5]. These ideas may be used for recruitment processes during job openings, but also when selecting participants for fellowships and cohort based trainings (which face similar constraints).
For more insights on how to enhance our hiring practises I recommend reading this article wrote by Abraham Rowe (@abrahamrowe), about the importance of being obsessed with recruitment.
This is one action I would encourage almost every AIS organization to apply outrightly.
As previously discussed, every player in the ecosystem is responsible to ensure a positive experience for candidates trying to contribute into the field, and guide them into next steps. The best moment to give this feedback is during the rejection emails sent after dismissing a candidate.
However, most AIS organizations are poorly or (directly neglecting) doing this. Consider these real examples from rejection letters:
- "In this initial review stage, we evaluated applications based on key criteria relevant to the role. Due to the high volume of applications, we are unable to offer personalized feedback at this time".
- "After careful review, we’re not able to offer you a place in this round. Given the number of applications, we're unable to provide individual feedback or reconsider decisions".
- "The selection process was highly competitive, (...) we've decided to move forward with other candidates this time, but we'd genuinely encourage you to apply again in the future".
- "This role attracted a strong field, and this isn't a reflection of your abilities. We'd encourage you to keep an eye on future roles".
Neither of these extracts from rejections letters help candidates, only encourage them to apply again without offering guidance on what to improve. And after dozens of times receiving the same experience, abandoning the field seems the next step.
However, due to the amount of applications hiring managers are actually unable to give personalize feedback to each candidate. How do we solve this? For instance, by giving scalable rejection feedback about the bar and objective criteria used to decide whether to move a candidate to next stage or not, plus giving some optional resources.
Disclaimer: This measure does not consist in sending again the requirements stated at the job description, but the actual bar used to separate candidates that advanced to the next stage versus those who did not. For instance:
"We are sorry to inform your application to the "Research Manager" role at our organization did not reach the next stage... This is the criteria and the bar we used to decide whether to advance candidates during this process, you might use it as a reference to assess how to improve future applications:
- They published at least 5 AI Safety papers at recognizable research organizations.
- +2 years of experience as a technical AI Safety researcher.
- 1 recognized leadership/coaching course or demonstrable experience at managing people.
- (...)
Also, we consider these resources could be helpful: X, Y, Z...". [6]
Consider 2 professionals who sent an application to this "Research Manager" role:
Again, these are not general outlines of a job description, but the objective criteria hiring managers used in that specific round after evaluating which talent available could advance to a next stage.
This means recruiters can elaborate a single rejection letter per stage to all applicants dismissed, being sent at once during an automation. This requires minimum effort if the hiring process is well designed, and provides an enormous positive effect to help those applicants in their job search, keeping them engaged within the AIS field.
Also, as a second-order effect the number of total applications gradually shrinks (being more manageable) and they become better quality, since these professionals are more informed and know to which roles apply and when they are prepared for them. So hiring managers also benefit from this practise in the long-run.
For more ideas on how to design rejection letters I recommend reading this article elaborated by Caleb Eli Price (@CalEA) . Also, for an excellent example of a rejection email giving scalable feedback you can read this post made by Gergő Gáspár (@gergo).
This is again an easy to implement action with massive potential returns.
Think about this: if many candidates are applying to a fellowship/job offer without being properly prepared, it is because they feel incentivized to apply. Either the role requirements were too general and vague, not clearly written and therefore wrongly interpreted, or applicants directly felt incentivized to apply anyway with phrases like "even if you feel you do not meet all the criteria above, we encourage you to send an application anyway!"[7]
A new organization without historic of applications number, neither tacit knowledge of how to execute a role, can allow vagueness and broader communication to attract as much talent as possible. And hence learn what the company actually needs to fill the role.
An established organization with years of experience in the field, opening the role a third time due to an expansion, cannot allow this kind of descriptions. For the sake of hiring specialists and qualified applicants, the job description should be much accurate and clear about the criteria and bar that past candidates needed to advance to next stages at that same role. [8]
In a sense, this is the next step after the scalable feedback in rejection letters discussed before. Now you use what you learned in past hiring processes to improve the filter before candidates send their applications. Saving their time and informing them about where to improve before applying.
This could be more controversial, but some companies may need it during a big expansion process.
If hardening job descriptions is not an effective enough measure to discourage less qualified applicants, the next step is to set a barrier to apply for the job. This could be as simple as a knowledge based test related the job position, which could be assessed through multiple choice questions aiming to a 80-95% success rate.
Some hiring managers could fear losing top candidates by setting more barriers. However, I would argue top candidates are the ones with high motivation to apply for a job they see fit for them, so they will not be pushed back by a test.
Again, implementing this measure demands an excellent understanding of the job requirements to be successfully accomplished. This should not be implemented by new companies unless thousands of non-qualified applications are expected.
Very situational, controversial and context dependent, but still worth mentioning. I would only recommend it if:
Then, a work trial as the application itself could be a potential solution. However, this is not the "2-4 hours paid work test" most AIS companies conduct as a standard in the industry. Rather would be a longer work trial resembling a critical function of the role, unpaid unless that work is used by the company, and sold as a project candidates can use both for learning and especially as a part of their portfolio.[9]
This was a hiring setting I used a few years ago, with high success.
I was working as Lead Project Manager / Chief of Staff for an e-learning company, offering a subscription based model for self-development and business books summaries. To meet the content production objectives I decided to hire 3 contractors as writers for the summaries.[10]
The CEO had a Spanish YouTube channel with +1 million subscribers and an email database of 34,000 leads. We planned to announce the job offer through those channels, expecting to receive thousands of applications (hard to be managed only by myself).
Instead of a plain form, I decided to design the application process as a work trial. Candidates would have to read the book "TED Talks: The official TED guide to public speaking", organize the book ideas and send a 3,000 - 4,000 words summary[11]. The expected work time would be around 20 hours and candidates were clearly informed to only be paid if they were among the 3 contractors selected, from which 1 of those summaries would be used.
I received about 45 applications instead of thousands of them. This allowed me to properly review each work trial by an objective criteria rubric I designed beforehand. The best candidates were advanced to a next stage which consisted of a 30 minutes job interview with me. And those who did not reach the bar received an email with scalable feedback about their summary, sent from 4 different feedback templates regarding most thematic mistakes.
And since the main function of the contractors was to write these summaries, the application process as a work trial was the best method to evaluate them.
Again, I am conscious this measure is very situational and could be controversial if applied as standard among the AIS industry, but it still worth considering for specific cases. [12]
AI Safety is in a critical moment of expansion that requires excellent hiring practises to be implemented. Risk-aversion to recruit, talent pipelines inefficiencies or feedback negligence could cost the field a big price in the coming future.
The intention of this post was not to criticize AIS hiring managers, but rather to be a wake-up call for the whole industry to improve how we treat our talent. The magnitude of the mission we want to accomplish requires investing in those who have the potential to be great professionals, but still need maturing.
Only then, we will be able to shape the course of how AI is developed. Changing regret for a sense of pride for the humans we supported, working in one of the highest impact causes of our whole history.
If at least one idea in this article was worth reading, consider sharing this post with a Hiring Manager, Chief of Staff, Grantmaker or Founder you know in the AI Safety field. Valuable posts in the EA forum are usually left unknown by the people who most good could do with that information. Your recommendation could be the act needed to change the course of AI Safety.
Not because hiring specialist are bad at their job at all, but due to system incentives and how the field hiring processes are designed. A great hiring professional inside a poor setting will probably do a deficient job because their capabilities are not able to be fully displayed in that context.
More often than only "Context", which could be measured by analysing use of language, critical thinking or EA/AIS knowledge during a job interview.
This is not to say that "Capabilities" or "Context and Motivation" are not important. They are key factors to properly perform in a job, ones that hiring managers also assess when considering someone. My argument is that even if these features are important, "Trust" seems the most relevant by how hiring managers tend to recruit, and one deciding factor they are not properly communicating to candidates.
We could define a "hiring success" for this case as incorporating someone that adequately accomplish the role functions, has a positive impact on the organizational culture and lasts over a 1 year period working for the company.
Therefore they will not be the "holy grail" nor the keystones to solve AIS talent bottlenecks. I will discuss some stronger but harder to apply measures in future posts.
As a counterargument someone could claim that is not possible to quantify all the criteria to inform the candidates about the decision. Even if this could be true in some cases, we still need to set objective bars to argument an application dismissal (e.g. "we prioritized early applicants"), or otherwise the hiring process is not well designed and likely to be biased. We need to convey more transparency on how these decisions are made.
Which is counterintuitive and dissonant for many applicants. Why encourage less qualified candidates to apply to an offer you expect (by previous experiences) to be flooded by forms? Again, candidates usually spend hours to craft a good application. To receive a rejection with messages like "this was a highly competitive setting" or "we received more applications than expected" seems inconsistent compared with similar statements in job descriptions.
Instead of "being familiar with AI Safety risks", say for instance "completed at least one AI Safety recognized course (BlueDot, Atlas...)". If a hiring manager fears leaving behind too much potential talent, they can write these same clear requirements but slightly below of where they set previous bars.
I know, I know, controversial. Seems like free work for companies, and I would not implement it as a standard for the industry. However, in specific situations could be the best solution, both for hiring managers and candidates. Especially if the conditions are clearly specified.
The official title of my job position was unclear, since I was part-time hired unexpectedly while working for other e-learning company of the same CEO. Probably closer to truth the title was "Lead Project Manager" doing also functions of a "Chief of Staff".
At that time LLMs were completely unknown. Nowadays, I would probably design the work trial differently to prevent LLM abuse.
In some way, current fellowships and in-person senior work trials already use this setup to assess a candidate before offering a permanent job position.
