Advice for AI safety university group organizers.

Acknowledgements

I collected most of these ideas in early January while attending OASIS 4.0, a three-day workshop for AI safety university group organizers in Berkeley, CA. Thank you to everyone who gave their input, and a special thanks to Jeremy Kintana, Neav Topaz, Tzu Kit Chan, and Chris Tardy for your feedback on earlier versions of this writeup. The contents of this post don't necessarily reflect the opinion of anyone but myself. All mistakes are my own. 

Summary

Given short timelines, AI safety university groups should:

• Prioritize grad students and skilled researchers who could have a meaningful impact within 2-3 years
• Run selective upskilling programs and small, high-context gatherings
• Protect researcher time for those doing impactful work
• Have a succession plan with clear documentation
• Invest in AI governance fieldbuilding and community wellbeing
• Be cautious and coordinated in advocacy efforts

Context

It seems like everyone’s timelines have been shortening.

One of the most respected voices on AI timelines is Ajeya Cotra. In November 2023, her median estimate for when 99% of currently fully remote jobs will be automatable was 13 years. In January, she wrote that AI progress on tasks that take human experts multiple hours is almost twice as fast as she expected. On timelines, she now defers to engineers at places like METR and Redwood who have hands-on experience with frontier AI.

Daniel Kokotajlo, another highly respected voice on timelines, has had a median timeline of 2027 for years. His probability distribution:^[1]

• 19% - 2025
• 19% - 2026
• 12% - 2027
• 6% - 2028
• 6% - 2029
• 4% - 2030
• 2% - 2031
• 2% - 2032
• 2% - 2033
• 2% - 2034
• 2% - 2035

When 99% of currently fully remote jobs are automatable, human AI safety researchers will be largely obsolete. If humans still control them, AIs will be the ones doing alignment research, not humans. Therefore, fieldbuilding is on a deadline.

This affects how AI safety university groups should allocate resources. If timelines are indeed likely to be short, here’s what I think AI safety university groups should do.

Resource Allocation

High-Priority Activities

1. Grad Student Outreach
   • Target grad students with relevant skills
   • Send them relevant papers
   • Invite them to your programs
2. Prof Outreach
   • Focus on those with relevant research areas
   • Consider visiting their office hours if they don’t reply to your emails
   • Ideas for prof engagement, from least to most involved:
     • Share relevant funding opportunities
     • Invite them to present at or attend your group’s events
     • Host moderated faculty panels and follow up individually
   • If a prof or grad student wants to learn more about AI safety research, you could recommend they request a call with Arkose
3. Upskilling Programs
   • Run selective programs for highly skilled participants
   • Steal from existing curricula (e.g., BlueDot Impact, ARENA)

Technical vs. Governance

Last year, 80,000 Hours replaced technical AI safety research with AI governance and policy as their top-ranked career path. See their reasoning here. Aside from being more neglected than technical AI safety, governance could also be more tractable given short timelines, especially for students.

Compared to established fields like physics, it’s faster to skill up in technical AI safety. Because it has less foundational research to study, it’s easier to get caught up to the cutting edge. This is even more true of AI governance — there’s still hardly any meaningful AI regulation anywhere in the world.

Don’t neglect governance. Consider:

• Directing technically-skilled students who can't contribute to technical safety within 3 years toward governance (or technical AI governance)
• Investing in outreach to those with skills relevant to AI governance, like policy grad students
• Responding to relevant government Notices of Inquiry or Requests for Comment (RFCs)^[2]
  • These likely go by a different name if you’re not in the US
  • Beyond potentially influencing important policy decisions, responding to RFCs is an excellent way to test your fit for policy work; policy think tanks spend lots of time on RFC responses, and their other outputs draw on similar skills
  • When writing a response, treat it as a focused research project targeting 1-3 sub-questions
  • You might find profs willing to supervise this work

Time Management

1. If you do impactful^[3] research, guard your time

   • Historically, the most impactful careers to come out of AI safety university groups have often been those of the organizers themselves
   • Therefore, don't trade full-time research to do more group organizing; prioritize your own development
   • Organize activities that complement your research (e.g., a paper reading group for papers you wanted to read anyway)
2. If you don’t do impactful research:
   • Seriously evaluate whether you could become an impactful researcher within 2-3 years
   • If yes, consider using your group to support that transition (e.g., organizing study groups for topics you want to learn)
   • If no, make sure your work aligns with your strategic model

Succession Planning

Even with short timelines, it might be important to avoid letting your group die. Why?

• New capable students arrive each year
• Timelines could be longer than you think
• It’s much easier to find a successor than for a future aspiring fieldbuilder to resurrect your group
• You can focus on activities that don’t heavily trade off against your other priorities
• Your group could be leveraged for future advocacy efforts (see below)
• Your group could be important for student wellbeing (see below)

Key elements of succession:

• Create and maintain documentation that details:
  • Role descriptions and responsibilities
  • Account passwords and assets
  • Past successes and failures
• Have every leader write transition documents
• Identify successors well in advance and gradually give them increasing responsibility
  • This allows you to transfer knowledge that isn't easily written down
• Make information discoverable without relying on current leadership

Warning signs of bad succession planning:

• Key info is siloed with individual leaders
• You prioritize current activities at the expense of documentation
• Interested students can’t find basic information about applying to leadership by themselves

Community Building

Recommendations

• Prioritize small, high-context gatherings over large events
• Build tight-knit networks of capable people

This has the benefit of attracting more capable people in the future — your average new group member is likely to be as capable as your average current group member.

Community Support and Wellbeing

Over time, students might become increasingly distressed about AI developments. Juggling academic pressure with the fear of you and everyone you love dying soon is not easy. Feeling a strong sense of urgency and powerlessness is a recipe for burnout. Combined with the sense that the people around you don’t understand your concerns, these ideas can be crushing.

As capabilities accelerate while timelines shrink, your group’s most important function could become emotional support; a healthy community is more likely to maintain the sustained focus and energy needed to be impactful. Some ideas:

1. As a leader, model healthy behaviors
   • Find the balance between urgency and wellbeing
   • Avoid burnout; this will seriously hurt you and your productivity
   • Normalize talking about emotional impacts
   • Set clear boundaries and take visible breaks
   • Share your coping strategies
2. Create opportunities for connection
   • Plan social events unrelated to AI
   • Build genuine friendships within the leadership team
   • Include time for personal check-ins during meetings
   • Remember that strong social bonds make hard times easier

Advocacy

A Word of Caution

1. If you’re funded by Open Philanthropy, your group is not allowed to engage in political or lobbying activities
   • Read the terms of your agreement carefully
   • Reach out to Open Philanthropy if you’re unsure whether something you're planning is allowed
2. AI safety is in danger of being left-coded
   • This survey found that EAs and alignment researchers identify as left-leaning and nonconservative
   • Given that most universities are also left-leaning, successful university group advocacy could unintentionally strengthen this association, hurting the chances of getting meaningful AI safety regulation from conservative policymakers
   • Yet historically, universities have played an important role in building momentum for social movements (e.g., environmentalism, civil rights)
   • Advocates should therefore frame AI safety as a broadly shared, nonpartisan issue
3. Contact Kairos leadership before doing anything that risks damaging the AI safety community’s credibility: contact@kairos-project.org 

Potential Activities

1. Opinion Pieces
   • Focus on well-researched, thoughtful perspectives
   • Start with university publications
   • Build toward high-impact venues
     • This seems difficult as an unknown student; one method would be to ghostwrite for a prominent professor at your school
2. Open Letters and Statements
   • Leverage academic credibility
   • AI safety university groups have helped get professors to sign the CAIS letter

Should You Stage a Protest?

You might consider it if:

1. A different (non-crazy) campus group is already planning to stage an AI protest
2. There was a severe warning shot
   • E.g., a rogue AI is known to have caused a global pandemic but lacked the ability to fully overpower humanity
   • This could dramatically increase public appetite for AI regulation
3. There is a coordinated (inter)national protest by a well-respected organization
   • This might be communicated by e.g., Kairos or the Center for AI Safety

But you should: 

• Have a clear (set of) policy objective(s)
• Be professional and non-violent
• First communicate with the broader AI safety community