Hiring Resources Playbook v0.1
For my SPAR project, which is to help improve the hiring process for new AI Safety organizations, and after the last couple of weeks having several interviews with hiring managers, there is an urgent bottleneck need to resolve this. All these organizations are actively growing and hiring, and will continue on this trend into the near future.
The Hiring Problem
I have focused my SPAR project to helping founders and hiring managers at new AI Safety organizations, typically small teams of <7 full time employees who are expected to grow 5x within the next year.
The reason why my focus is with these organizations, is because I see the greatest pain in their hiring process (usually broken or none existent) and if resolved, will allow these organizations to have the greatest impact in improving their hiring process, not only with matching them efficiently with the ideal candidate, but also allowing them to overcome these barriers of hiring, and returning to continuing their main focus work on AI Safety. I believe that these smallest and newest organizations have not only the most growth potential in terms of their project's impacts, but the new people that they hire can help accelerate this work, greater than at a more established organization.
These new AI Safety organizations are dealing with two issues that my research indicates that are contributing to this hiring and growth bottlenecks. These are:
- They are trying to grow very quickly, and they fully understand the urgency that there is a very short window for AI Safety alignment before ASI, which all AI Safety organizations understand. Compound this with while in super drive to work on their research or project, that as a small or new organization, they are also trying to dramatically build their organization, work on their projects research while also compounding an increase in hiring to catch up.
- This leads to a bottleneck of their own design. Building and Growing are hard to do even for a larger organization, and with the intensity and the closing window for obtaining AI alignment it is even more urgent. With more AI Safety organizations hiring right now, the application volume will rise.
The Work test and Scoring Problem
The biggest issue I have discovered with my interviews of hiring managers of new AI Safety organizations, is that there is a discrepancy on who writes the tests, who scores them, and can the review tell a good work for merely OK work?
Here’s what I’ve discovered:
- Small hiring teams that review tests build upon a mixture of older work tests, with no clear check on reviewer expertise or no-researcher roles.
- Hiring for specialized roles (CFO) and unsure on how they will verify a good response in a work test
- Few organizations have used their network to use trusted outside experts to design and evaluate the tests and conduct interviews, but this is unique in new AI Safety organizations.
- Most candidates, due to these inefficiencies, are waiting months or more, and the hiring teams worry about losing strong candidates to other AI Safety organizations.
The outside-expert model
One very promising model that has emerged from my interviews is to bring in external domain experts to handle work tests, interviews and evaluations of candidates, and particularly for specialized roles for CFO, General/Legal Council, Communications roles, and even HR roles.
In early-stage teams without deep specialized knowledge (outside of research and engineering) or existing functional leads across every department, using targeted outside expertise allows hiring managers to maintain the evaluation rigor, obtain corporate credibility, without sacrificing speed and even increases their candidate experience.
In Practice this looks like:
- The hiring team can conduct the initial pass on communication clarity, AI Safety alignment, cultural understanding, problem-solving, and strategic reasoning, while the expert can evaluate the candidate's core technical and operational substance
- How to Find an Outside Expert: Ask widely across your existing founder and advisor networks. Founders need to be actively outreaching in their trusted circles to discover specialized experts. Expanding their network connections outside of the AI Safety community also allows for greater connection opportunities.
- Compensation: Pay experts for their time. Making the investment in these strategic roles (that could make or break a company if you get the right person hired) and therefore, paying for a highly knowledgeable expert's time is key for your organization's effectiveness and its a high ROI (return on investment) use of your capital.
Expert Workflow Across The Stages:
- Test Design: Craft realistic, domain-specific work samples and evaluation criteria.
- Scoring: Grade candidate submissions against a standardized rubric.
- Interviewing: Participate in or lead the technical and functional evaluation round.
Stage-by-Stage Checklist
- Role Definition: Hiring teams and founders need to learn 20–50% of the core role mechanics yourself, before you start the process of fulfilling your new job role. This enables you to evaluate candidates effectively and build immediate credibility during technical discussions
- Sourcing: Source broadly beyond immediate circles . While network referrals frequently yield high-quality candidates, relying strictly on them severely restricts long-term talent pool diversity and pipeline scale. And with increased growth of the expanding AI Safety community, there will be a greater need for more specialized mid to senior roles that are not as existent in the current AI Safety candidate pool.
- Applications: Keep the initial application forms brief and frictionless, and consider skipping formal application forms entirely for warm referrals.
- Screening: Evaluate candidate alignment and learning velocity dynamically (ie. sending new reading materials or resources between interview rounds and assessing the quality of their follow-up questions). Avoid rejecting borderline candidates purely on paper; AI-polished resumes or jargon-heavy applications can mask exceptionally strong non-traditional contributors.
- Work Tests: Always pay candidates for completed work tests, clearly explain the scoring criteria beforehand, and offer flexible deadlines for currently employed applicants
- Interviews: Structure evaluation into two dedicated sessions: an internal culture and alignment interview, followed by a technical interview led by the outside domain expert. This will allow for the hiring team to get a well rounded picture of organizational fit and skill assessment and validated expertise knowledge.
- Trial: Run a short, paid, in-person trial period before extending a formal offer. This is common in most organizations that I interviewed. Common time was three days, one organization conducted two weeks of on-site work trials. Understanding that this takes a dedicated team to prioritize the hiring process, in addition over their other possible responsibilities.
- Close: Re-evaluate your closing strategies against broader startup/other AI Safety org benchmarks. Additionally, using a startup head of talent expert to assist with covering competitive closing mechanics is worth reviewing and investing in, prior to creating a final offer letter.
Where the Failure Modes Are
- Lagging Work-Test Turnaround: Extended review windows lead directly to candidate drop-off and severe reputational risk among top-tier talent
- Tooling Fragmentation: Managing hiring across isolated Google Docs, disparate AI grading tools, and standalone Applicant Tracking Systems (ATS) introduces administrative friction and lost candidate data
- Over-Reliance on Referrals: Depending solely on immediate networks creates a sharp ceiling on candidate reach. Leaning heavily on referrals if your application volumes spike, though this risks narrowing your pipeline diversity, so you need a balanced approach.
- Out-of-Domain Founder Interviews: A recurring pattern observed across teams is founders directly interviewing candidates for specialized roles (such as Finance or Ops) where the founder may lack the deep functional domain expertise, which could lead to narrowed subjective or misaligned hiring decision
What I Have Not Yet Validated
To maintain a research rigor, I have several core assumptions in this playbook that require empirical testing:
- Outside Expert Efficacy: Whether engaging outside domain experts will measurably improve work-test grading accuracy or long-term candidate retention remains unproven.
- Non-Research Assessor Qualification: It is unverified whether internal research staff can accurately grade operational, legal, or financial work tests without external guidance.
- Scale Capacity: How early-stage operations will handle 5x to 10x increases in candidate application volume without breaking existing review workflows.
My Hypothesis to Test First:
Introducing a standardized rubric and an outside expert reviewer for non-research roles will reduce work-test grading turnaround time by 50% while increasing hiring manager confidence scores in candidate selection.
Call for Pilot Partners
I am actively seeking 3 to 5 early-stage AI Safety organizations willing to pilot this framework or provide structured feedback on the checklist.
What Pilot Partners Receive:
- A customized v1 work-test template and grading rubric for upcoming non-research or specialized roles.
- Hands-on support in identifying, onboarding, and coordinating outside domain evaluators.
What We Will Measure:
- Total calendar days required to review and turn around work tests.
- Evaluator confidence scores during candidate assessment.
- Post-hire operational performance as evaluated by the host organization, at 30-90-120 days after start date.
How to Reach Out, and How You Can Help:
If you lead an early-stage AI Safety team and want to streamline your candidate evaluation process, connect with me directly to join the pilot cohort.
Also, my intent is to continue to improve this process and Playbook with practical hiring processes for the AI Safety community, and specifically, for the new organizations that I believe have the greatest impact if they solve their hiring bottlenecks and allow them to focus on their projects and research.
If I have misstated or missed important steps to include this process, I want to hear from you, or comment. Likewise if you are in support of this project and want to help contribute. I’m always open to helping and talking to anyone in the community. Please feel free to reach out to me at: achartersmith@gmail.com
