What’s the case for hiring more grantmakers rather than lowering the rigorousness bar?
This doesn't fully address your question, but as far as I* can tell, the AIS grantmaking ecosystem is lowering the bar for how rigorous grant investigations need to be relative to a year ago.
*Annoying caveat: this is just my subjective opinion, I might lack visibility on things, yadda yadda yadda.
He answered that grantmakers needed to be "Relatively senior (6+ years of work experience?) AI-safety-focused people."
FWIW I don't actually agree with this interpretation, but I see how my comment could have been confusing.
Let me try to clarify:
I don't think grantmakers need to be senior. As Luke points out, CG has hired a bunch of people out of undergrad/grad school who ended up contributing pretty quickly.
In any case, I do think senior people are much more of a bottleneck, for reasons including what Luke mentioned here about management capacity being scarce.
I still agree with the claim that there are "many bright young people who want to work for EA charities", but I should have clarified in my comment that I don't think being a bright young person is sufficient.
If you look at the "Call to action" section of my post, I list a few characteristics that I think are important for being a good grantmaker. In my experience when hiring grantmakers, there are a lot of people who I would consider bright and impact-focused that nevertheless lack a number of these important traits.
Agreed there are many bright young people who want to work for EA charities. CG doesn't seem to have that much trouble getting these folks to apply (especially now that there are opportunities to test your fit via something like the Astra fellowship).
Relatively senior (6+ years of work experience?) AI-safety-focused people who can manage others, scope out programmatic foci and strategies, and build teams are much more of a bottleneck. Of course, this is true throughout the AI safety ecosystem, but it seems particularly tricky for grantmaking, and makes it harder to hire a bunch of eager young generalists.
Speculation: Maybe people who are more experienced are less open to trying new types of work, and grantmaking is fairly unusual?
Grantmaking means giving someone decent amounts of influence over large amounts of money, and orgs are (reasonably!) pretty cautious about extending that authority.
Compare this to something like research, where you can hire a researcher and have them produce work that gets reviewed before it goes anywhere.
With a grantmaker, the stakes of a bad judgment call are higher and can be harder to quickly review if you're a time-constrained manager.
This makes orgs slower to hire and slower to ramp people up to full autonomy than they would be for other roles.
CG has probably just straight up made mistakes around hiring too slowly in the past (and possibly the present). Ideally a single organization making a mistake like this wouldn't affect the ecosystem so much, but CG is such a large portion of the total amount of grantmaking done in the AI safety ecosystem ATM.
As someone not working in the AI safety space, I'm intrigued by your opinions as to what extent grant making within AI safety is similar to and different from grant making within other cause priority areas, for example animal advocacy and global development and health?
It's hard for me to say about what these differences look like outside of CG, but one thing that comes to mind is that GHW and animal welfare grantmaking is more based on quantitative modelling and BOTECs (though sometimes we use BOTECs on the GCR side of things).
My sense from reading the post is that those areas may be relatively less neglected, with fewer opportunities for identifying opportunities with outsized impact returns on investment. Do you think that is a reasonable assumption to be making?
It depends on how you define "neglected". Like, in terms of EA focus and talent, they're probably more neglected than AI safety/catastrophic risks. In terms of total $ spent by society at large, GHW is far less neglected than AI safety, which is in turn far less neglected than FAW.
This is kind of a lame answer, but whether AI safety has more or fewer outsized ROI opportunities really depends on your worldview. IMO, both spaces have a ton of opportunity. If I woke up tomorrow and decided that AI safety was no longer important (or I didn't buy that worldview anymore), I'd be extremely excited about the vast number of opportunities to make global health and farmed animal welfare better.
Currently it seems like each grantmaker is (on average) responsible for ~$10m/y. One question I think about sometimes: how will # of grantmakers scale as more $ go towards AI safety funding? If funding is eg 3x'ing year-over-year, it's unclear whether we're currently training up that number of grantmakers.
My vibes-based sense is that at least currently, the amount of philanthropic capital that could go towards AI safety projects is growing quite a bit faster than the number of grantmakers. I'm pretty worried about this.
Taking a look at CG:
Per this, the number of GCR program staff at CG only grew 2x from 2019 to 2022.
Quickly looking at archives of the team list from EOTY 2022 and EOTY 2025, it looks like the growth rate of program staff over that period was roughly 2 to 2.5x.
Another question might be: what is a good ratio of # of grantmakers to # of direct work? I'd ballpark there to be ~1000 fulltime AIS direct workers; does a 20:1 ratio seem high, low, or just right?
I think the ratio framing is a bit tricky and depends on a lot of other variables (for instance, how mature the field is, how many promising ideas are floating around the memesphere, how good AIs are at doing direct work, how much philanthropic capital there is, etc). The other thing is that the number of direct workers is itself downstream of grantmaker capacity.
Thanks for your work on this, Will and Max! Suffice to say this is pretty cool.
However, I'm a bit disheartened that you pushed the frontier without formally releasing an RSP ("Responsible-ish Scaling Policy").
So let me make this unambiguous: Do you commit to pausing the scaling and/or delaying the deployment of new automated macrostrategy researchers whenever your scaling ability outstrips your ability to comply with the safety procedures for the corresponding PSL (Philosophical Safety Level)?
I bet a number of generalist EAs (people who are good at operations, conceptual research / analysis, writing, generally getting shit done) should probably switch from working on AI safety and policy to working on biosecurity on the current margin.
While AI risk is a lot more important overall (on my views there's ~20-30% x-risk from AI vs ~1-3% from bio), it seems like bio is a lot more neglected right now and there's a lot of pretty straightforward object-level work to do that could take a big bite out of the problem (something that's much harder to come by in AI, especially outside of technical safety).
If you're a generalist working on AI because it's the most important thing, I'd seriously consider making the switch. A good place to start could be applying to work with my colleague ASB to help our bio team seed and scale organizations working on stuff like pathogen detection, PPE stockpiling, and sterilization tech. IMO switching should be especially appealing if:
You find yourself unsatisfied by how murky the theories of change are in AI world and how hard it is to feel good about whether your work is actually important and net positive
You have a hard sciences or engineering background, especially mechanical engineering, materials science, physics, etc (or of course a background in biology, though that's less necessary/relevant than you may assume!)
You want a vibe of solving technical problems with strong feedback loops rather than a vibe of doing communications and politics, but you're not a good fit for ML research
To be clear, bio is definitely not my lane and I don't have super deep thinking on this topic beyond what I'm sharing in this quick take (and I'm partly deferring to others on the overall size of bio risk). But from my zoomed-out view, the problem seems both very real and refreshingly tractable.
Like Ajeya, I haven't thought about this a ton. But I do feel quite confident in recommending that generalist EAs — especially the "get shit done" kind — at least strongly consider working on biosecurity if they're looking for their next thing.
I had the enormous privilege of working at Giving What We Can back in 2021, which was one of my first introductions to the EA community. Needless to say, this experience was formative for my personal journey with effective altruism. I consider Luke an integral part of this.
I can honestly say that I've worked with some incredible and brilliant people during my short career, but Luke has really stood out to me as someone who embodies virtue, grace, kindness, compassion, selflessness, and a relentless drive to have a large positive impact on the world.
Luke: thank you for everything you've done for both GWWC and the world, and for the incredible impact that I'm confident you will continue to have in the future. I'm sad to imagine a GWWC without you at the helm, but I'm excited to see the great things you'll end up doing down the line after you've had some very well deserved time with your family.
This doesn't fully address your question, but as far as I* can tell, the AIS grantmaking ecosystem is lowering the bar for how rigorous grant investigations need to be relative to a year ago.
*Annoying caveat: this is just my subjective opinion, I might lack visibility on things, yadda yadda yadda.
Note for those who haven't met Lewis in person: he is very tall
FWIW I don't actually agree with this interpretation, but I see how my comment could have been confusing.
Let me try to clarify:
Some thoughts:
Thanks for the comment!
It's hard for me to say about what these differences look like outside of CG, but one thing that comes to mind is that GHW and animal welfare grantmaking is more based on quantitative modelling and BOTECs (though sometimes we use BOTECs on the GCR side of things).
It depends on how you define "neglected". Like, in terms of EA focus and talent, they're probably more neglected than AI safety/catastrophic risks. In terms of total $ spent by society at large, GHW is far less neglected than AI safety, which is in turn far less neglected than FAW.
This is kind of a lame answer, but whether AI safety has more or fewer outsized ROI opportunities really depends on your worldview. IMO, both spaces have a ton of opportunity. If I woke up tomorrow and decided that AI safety was no longer important (or I didn't buy that worldview anymore), I'd be extremely excited about the vast number of opportunities to make global health and farmed animal welfare better.
My vibes-based sense is that at least currently, the amount of philanthropic capital that could go towards AI safety projects is growing quite a bit faster than the number of grantmakers. I'm pretty worried about this.
Taking a look at CG:
I think the ratio framing is a bit tricky and depends on a lot of other variables (for instance, how mature the field is, how many promising ideas are floating around the memesphere, how good AIs are at doing direct work, how much philanthropic capital there is, etc). The other thing is that the number of direct workers is itself downstream of grantmaker capacity.
Thanks for your work on this, Will and Max! Suffice to say this is pretty cool.
However, I'm a bit disheartened that you pushed the frontier without formally releasing an RSP ("Responsible-ish Scaling Policy").
So let me make this unambiguous: Do you commit to pausing the scaling and/or delaying the deployment of new automated macrostrategy researchers whenever your scaling ability outstrips your ability to comply with the safety procedures for the corresponding PSL (Philosophical Safety Level)?
Ajeya Cotra writes:
Like Ajeya, I haven't thought about this a ton. But I do feel quite confident in recommending that generalist EAs — especially the "get shit done" kind — at least strongly consider working on biosecurity if they're looking for their next thing.
People who participate in talent development programs can go on to work in a variety of roles outside of the government and AI companies.
I had the enormous privilege of working at Giving What We Can back in 2021, which was one of my first introductions to the EA community. Needless to say, this experience was formative for my personal journey with effective altruism. I consider Luke an integral part of this.
I can honestly say that I've worked with some incredible and brilliant people during my short career, but Luke has really stood out to me as someone who embodies virtue, grace, kindness, compassion, selflessness, and a relentless drive to have a large positive impact on the world.
Luke: thank you for everything you've done for both GWWC and the world, and for the incredible impact that I'm confident you will continue to have in the future. I'm sad to imagine a GWWC without you at the helm, but I'm excited to see the great things you'll end up doing down the line after you've had some very well deserved time with your family.