I think the EA/AIS community has been remarkably successful at recruiting and deploying talent to address AI misalignment and misuse risks. I'd like to reverse engineer their approach to see if we can apply a similar recipe to other "grand challenges" emerging from rapid AI progress.
What's missing from my naive explanation for the recent success of the AIS community?
- Conceptual Foundations (pre-2016): Early conceptual work established the importance of AI Safety and clarified the most valuable lines of action
- Skill and career capital development (~2016-2021): Those producing conceptual work, along with their most engaged supporters, began doing direct work in the area. While much pre-LLM work may not have been directly impactful, it built crucial skills and career capital through organizations, connections, experience running successful programs, etc.
- Examples: Open Philanthropy's AI risks program, EA community consolidation (EAGs, university groups), AGI Safety Fundamentals course, CSET, CHAI, SERI MATS
- Talent conversion (2022 onwards): When AI gained broader attention and impact opportunities became clearer, there was already a stock of arguments for AI Safety's importance and a community ready to address talent pipeline bottlenecks and enroll more people in useful work.
- Examples: Open Philanthropy's RFPs, MATS, BlueDot, Anthropic and DeepMind AIS teams, ARENA, AI-risk-focused events within EA
In this simplified story, if we want to draw more talent to newly formulated AI challenges, the most important things are:
- Broadcasting the case for working in the area and identifying concrete lines of action
- Developing a base of early practitioners with the skills and career capital needed to pave the way for future work
- Creating a large influx of people with initial inclination to work in the area (e.g., from the EA community)
that's just false. alignment community even to this day is significantly less then 1000 people worldwide. i would not call that a success. i'm not blaming the people involved. they've done what they could. lets just not deceive ourselves here
Well, AI sentience probably has <100 people and I'd be joyful if that increased to 1k. I'm still interested even if the question is "How can we fail less misserably at creating a talent pipeline?"