This post was prompted by three other posts on the EA forum. A recent post raises the alarm about abuse of power and relationships in the EA community. An earlier post suggested that the EA community welcomes shallow critiques but is less receptive to deep critiques. Dustin Moskovitz has recently mentioned that the EA forum functions as a conflict of interest appeals board for Open Philanthropy. Yet on the EA forum, there doesn’t seem to be many specific criticisms of Open Philanthropy.
For the sake of epistemics, I wanted to create this post and invite individuals to voice any issues they may have with Open Philanthropy or propose potential solutions.
I’ll start by discussing the funding dynamics within the field of technical alignment (alignment theory, applied alignment), with a particular focus on Open Philanthropy.
In the past two years, the technical alignment organisations which have received substantial funding include:
All of these organisations are situated in the San Francisco Bay Area. Although many people are thinking about the alignment problem, there is much less funding for technical alignment researchers for other locations (e.g., the east coast of the US, the UK, or other parts of Europe).
This collectively indicates that, all else being equal, having strong or intimate connections with employees of Open Philanthropy greatly enhances the chances of having funding, and it seems almost necessary. As a concerned EA, this seems incredibly alarming and in need of significant reform. Residency in the San Francisco Bay Area is also a must. A skeptical perspective would be that Open Philanthropy allocates its resources to those with the most political access. Since it's hard to solve the alignment problem, the only people grantmakers end up trusting to do so are those who are very close to them.
This is a problem with Open Philanthropy’s design and processes, and points to the biases of the technical alignment grantmakers and decision makers. This seems almost inevitable given (1) community norms around conflicts of interest and (2) Open Philanthropy’s strong centralization of power. This is not to say that any specific individual is to blame. Instead processes, structure, and norms are more useful to direct reforms towards.
Right now, even if a highly respected alignment researcher thinks what you do is extremely valuable, the decision ultimately can be blocked by an Open Philanthropy grantmaker, which could cause people to leave alignment altogether.
One common suggestion involves making the grantmaking process more democratic or less centralised. For example, the “regranting” approach has been successful for other grantmakers. This involves selecting a large pool of grantmakers or regrantors who have the autonomy to make their own decisions. With more grantmakers, there is less potential for Goodharting by individuals and reduces the likelihood of funding only those who are known best by a few Open Philanthropy staff. Additionally, Open Philanthropy can still choose regrantors who are more aligned with EA values or have previously demonstrated good judgement. A smaller thing that could help is indicating what proportion of funding in each specific area goes to organisations where there are, or have been, intimate relationships between organisation leadership and Open Philanthropy grantmakers.
It seems vital for community health to be able to speak about these sorts of issues openly. What criticisms of Open Philanthropy are important to share, or what are other ways they should improve?
Your post does not actually say this, but when I read it I thought you were saying that these are all the organizations that have received major funding in technical alignment. I think it would have been clearer if you had said "include the following organizations based in the San Francisco Bay Area:" to make it clearer you're discussing a subset.
Anyway, here are the public numbers, for those curious, of $1 million+ grants in technical AI safety in 2021 and 2022 (ordered by total size) made by Open Philanthropy:
The Alignment Research Center received much less: $265,000.
There isn't actually any public grant saying that Open Phil funded Anthropic. However, that isn't to say that they couldn't have made a non-public grant. It was public that FTX funded Anthropic.
Based on spending some time in Berkeley, I think a more accurate way to describe this is as follows:
People who care about AI safety and are involved in EA tend to move to Berkeley because that is where everyone else is. It really can increase your productivity if you can easily interact with others working in your field and know what is going on, or so the established wisdom goes. The people who have been around the longest are often leading research organizations or are grantmakers at Open Phil. They go to the same parties, have the same friends, work in the same offices, and often spend nearly all of their time working with little time to socialize with anyone outside their community. Unless they make a special effort to avoid dating anyone in their social community, they may end up dating a grantmaker.
If we want these conflicts of interest to go away, we could try simply saying it should be a norm for Open Phil not to grant to organizations with possible conflicts of interest. But knowing the Berkeley social scene, this means that many Open Phil grantmakers wouldn't be able to date anyone in their social circles, since basically everyone in their social circles is receiving money from Open Phil.
The real question is as you say one of structure: whether so many of the EA-aligned AI safety organizations should be headquartered in close proximity and whether EAs should live together and be friends with basically only other EAs. That's the dynamic that created the conflicts. I don't think the answer to this is extremely obvious, but I don't really feel like trying to argue both sides of it right now.
It's possibly true that regrantors would reduce this effect in grantmaking, because you could designate regrantors in other places or who have different friends. But my suspicion would be that regrantors would by default be the same people who are already receiving grants.
I was looking into this topic, and found this source:
Speculating, conditional on the pitchbook data being correct, I don't think that Moskovitz funded Anthropic because his object level beliefs about their value or because they're such good pals, rather I'm guessing he received a recommendation from Open Philanthropy, even if Open Philanthropy wasn't the vehicle he used to transfer the funds.
Also note that Luke Muehlhauser is part of the board of Anthropic; see footnote 4 here.