The Vultures Are Circling

This post uses an alarmist tone to trigger emotions ("the vultures are circling"). I'd like to see more light and less heat. How common is this? What's the evidence?

People have strong aversions to cheating and corruption, which is largely a good thing - but it can also lead to conversations on such issues getting overly emotional in a way that's not helpful.

Fwiw, anecdotally my impression is that a more common problem is that people engage in motivated reasoning to justify projects that aren't very good, and that they just haven't thought through their projects very carefully. In my experience, that's more common than outright, deliberate fraud - but the latter may get more attention since it's more emotionally salient (see my other comment). But this is just my impression, and it's possible that it's outdated. And I do of course think that EA should be on its guard against fraud.

Would really appreciate links to Twitter threads or any other publicly available versions of these conversations. Appreciate you reporting what you’ve seen but I haven’t heard any of these conversations myself.

Like other commenters, to back-up the tone of this piece, I'd want to see further evidence of these kinds of conversations (e.g., which online circles are you hearing this in?). 

That said, it's pretty clear that the funding available is very large, and it'd be surprising if that news didn't get out. Even in wealthy countries, becoming a community builder in effective altruism might just be one of the most profitable jobs for students or early-career professionals. I'm not saying it shouldn't be, but I'd be surprised if there weren't (eventually) conversations like the ones you described. And even if I think "the vultures are circling" is a little alarmist right now, I appreciate the post pointing to this issue.

On that issue: I agree with your suggestions of "what not to do" -- I think these knee-jerk reactions could easily cause bigger problems than they solve. But what are we to do? What potential damage could there be if the kind of behaviour you described did become substantially more prevalent?

Here's one of my concerns: we might lose something that makes EA pretty special right now. I'm an early-career employee who just started working at an EA org . And something that's struck me is just how much I can trust (and feel trusted by) people working on completely different things in other organisations. 

I'm constantly describing parts of my work environment to friends and family outside of EA, and something I often have to repeat is that "Oh no, I don't work with them -- they're a totally different legal entity -- it's just that we really want to cooperate with each other because we share (or respect the differences in) each other's values". If I had to start second-guessing what people's motives were, I'm pretty sure I wouldn't feel able to trust so easily. And that'd be pretty sad.

So what to do? I’d like to note that some of the knee jerk reactions when hearing of the problem are examples of things not to do.

This seems overly quick to rule out a large class of potential responses. Assuming there are (or will be) more "vultures," it's not clear to me that the arguments against these "things not to do" are solid. I have these hesitations (among others) [edited for clarity and to add the last two]:

• "The rationale for giving out high risk grants stands and hasn’t changed."
  • Sure, but the average level of risk has increased. So accepting the same level of risk means being more selective.
• "decreasing the riskiness of the grants just means we backslide into becoming like any other risk averse institution."
  • Even if we put aside the previous point, riskiness can go down without becoming as low as that of typical risk-averse institutions.
• "Increasing purity tests. [...] As a community that values good epidemics, having a purity test on whether or not this person agrees with the EA consensus on [insert topic here] is a death blow to the current very good MO."
  • There are other costly signals the community could use.
• "So not funding young people means this type of talent and potential is wasted. Let's not do that."
  • Just because something has downsides doesn't mean we shouldn't do it; maybe it's worth it to waste some talent to avoid many vultures. (I'm not saying that's the case, just that more consideration would be helpful.)

I feel like this community was never meant to scale. There is little to no internal structure, and like others have said, so much of this community relies on trust. I don't think this is just an issue of "vultures", it will also be an issue of internal politics and nepotism. 

To me the issue isn't primarily about grantmaking. If you are a good grantmaker, you should see when people's proposals aren't super logical or aligned with EA reasoning. More people trying to get big grants is mostly a good thing, even if many are trying to trick us into giving free money. I think the much larger issue is about status/internal politics, where there is no specific moment if you can decide how aligned someone is. 

But first to give some evidence of vultures, I have already seen multiple people in the periphery of my life submit apps to EAGs who literally don't even plan on going to the conferences, and are just using this as a chance to get a free vacation. I feel sorry to say that they may have heard of EA because of me. More than that, I get the sense that a decent contingent of the people at EAGx Boston came primarily for networking purposes(and I don't mean networking so the can be more effective altruists). At the scale we are at right now, this seems fine, but I seriously think this could blow up quicker than we realize.

Speaking to the internal politics,  I believe we should randomly anonymize the names on the on the forum every few days and see if certain things are correlated with getting more upvotes (more followers on twitter, a job at a prestigous org, etc.). My intuition has been that having a job at a top EA org means 100-500% more upvotes on your posts here, hell even the meme page. Is this what we want? The more people who join for networking purposes, potentially the worse these effects become. That could entail more bias. 

I post (relatively) anonymously on twitter, and the amount of (IMO) valid comments I make that don't get responded to makes me worry we are not as different from normal people as we claim, just having intellectual jousts where we want to seem smart among the other high status people. To be fair this is an amazing community and I trust almost everyone here more than almost anyone not in this community to try to be fair about these things.

I  get the sense (probably because this is often going on the back of my mind), that many people are in fact simply optimizing for status in this group, not positive impact as they define it themself. Of course status in this community is associated with positive impact, BUT as defined by the TOP people in the community. Could this be why the top causes haven't changed much? I don't feel strongly about this, but it's worth considering.

As a former group organizer, there is a strong tension between doing what you think is best for the community vs for yourself. Here is an example: To build resilience for your group, you should try to get the people who might run the group after you leave to run events/retreats/network with other group organizers, so they are more committed,  have practice, and have a network built up. But you get more clout if you run retreats, if you network with other group organizers, etc. It takes an extremely unselfish person to not just default to not delgating a ton of stuff, in no small part for the clout benefits. This tension exists now, so I'm not claiming this would only result from the influx of money, but now that organizers can get jobs after they graduate school, expect this to become a bigger issue.

P.S. If the community isn't meant to scale, then individual choices like vegetarianism are not justified within our own worldview.

It's already happening. The two people I vetted for FTX Future Fund grant application didn't pass muster but, I dare say, they've perfected the worst version of "isomorphic mimicry" (for the lack of a better phrase). I'm not sure I can share details of this private conversation but it's good someone is pointing it out. 

Just my impression based on anecdotes, but I've heard about more people from outside of the  community trying to get FTX grants than I've noticed in the past (e.g. with Open Phil). The word seems to have 'gotten out' to a greater degree than before.

To bring in some numbers, FTX had a huge number of applications, so most of these must have come from outside the current community. And it seemed like the number was greater than expected.

To bring in some theory, it would make sense this is happening based on SBF's fame and the strategy they're taking (rapid, low overhead grants for a wider range of things).

So overall it seems plausible to me this is now happening to a greater degree than in the past, but I'm very unsure how much more, and how much of a problem it is. It's probably semi-inevitable as you get bigger.

I downvoted this post because it doesn't present any evidence to back up its claims. Frankly I also foudn the tone off-putting ("vultures"? really?) and the structure confusing. 

I also think it underestimates the extent to which the following things are noticeable to grant evaluators. I reckon they'll usually be able to tell when applicants (1) don't really understand or care about x-risks, (2) don't really understand or care about EA, (3) are lying about what they'll spend the money on, or (4) have a theory of change that doesn't make sense. Of course grant applicants tailor their application to what they think the funder cares about. But it's hard to fake it, especially when questioned.

Also, something like the Atlas Fellowship is not "easy money". Applicants will be competing against extremely talented and impressive people from all over the world. I don't think the "bar" for getting funding for EA projects has fallen as much as  this post, and some of the comments on this post, seem to assume.

Increasing purity tests

If you call them 'purity tests' that has a bad connotation.

whether or not this person agrees with the EA consensus on [insert topic here]

Obviously that test would be terrible for the intellectual and epistemic environment of EA. We shouldn't screen on 'whether agrees with outcome'...

But it is reasonable to consider 'epistimic virtues' as inputs ... 'whether someone engages in honest debate, their reasoning is transparent' ... something less stringent than the (CEA principles)[https://www.centreforeffectivealtruism.org/ceas-guiding-principles] perhaps.

I also think considerations like 'does this person have a track record of engaging with EA and EA-adjacent activities before applying for this' should yield some good signaling/screening value.

(I see Mauricio made a similar point)

Thanks for the post. I share your concerns, and I even enjoy the kind of alarmist tone. However, I think some possible objections would be:
a) Perhaps job applications are more effective at marketing EA than other strategies. Publish a good job offer, and you can make dozens or hundreds of talented and motivated people dive into EA concepts.

b) Maybe false positive rates are increasing, but what about recall? It's all about trade-offs, right? There are probably many people with EA potential out there; how many vultures are you willing to let in to attract them?

c) I don't have a problem with "effective" vultures. If they can, e.g., solve the alignment problem or fill the operational needs of an EA organization, does it matter a lot that they are just building career capital?

“You should apply for [insert EA grant], all I had to do was pretend to care about x, and I got $$!”

I can speak of one EA institution, which I will not name, that suffers from this. Math and cognitive science majors can get a little too  far in EA circles just by mumbling something about AI Safety, and not delivering any actual interfacing with the literature or the community. 

So, thanks for posting. 

Atlas Fellowship cofounder here. Just saw this article. Currently running a workshop, so may get back with a response in a few days.

For now, I wanted to point out that the $50,000 scholarship is for educational purposes only. (If it says otherwise anywhere, let me know.)

I might make it clearer that your bullet points are what you recommend people not do. I was skimming and at first and was close to taking away the opposite of what you intended.

Thank you for writing this post. The discussion and critiques brought up are important and valuable, and I just want to say that I'm grateful you put this out there, since I've been very worried about the same things. 

I'll hop on the "I'd love to see sources" train to a certain extent, but honestly we don't really need them. If this is happening it's super important, and even if it isn't happening right now it'll probably start happening somewhat soon. We should have a plan for this.

Vultures ≈ Death, typically

EA = Not dead, quite the opposite in fact

I think we should be careful about how we communicate. Maybe instead of just saying that there is "lots of funding available" we should clarify that we mean that there's lots of funding available for people who can deliver. That is less likely to draw in vultures.

As a community that values good epidemics

good epistemics?

Thanks for posting about this; I had no idea this was happening to a significant extent.

I think it is important to keep in mind that we are not very funding constrained. It may be ok to have some false positives, false negatives may often be worse, so I wouldn't be too careful.

I think grantmaking is probably still too reluctant to fund stuff that has an unlikely chance of high impact, especially if they are uncertain because the people aren't EAs.
For example, I told a very exceptional student (who has like 1 in a million problem solving capability) to apply for Atlas fellowship, although I don't know him well, because from my limited knowledge it increases the chance that he will work on alignment from 10% to 20-25%, and the $50k are easily worth it.

Though of course having more false positives causes more people that only pretend to do sth good to apply, which isn't easy to handle for our current limited number of grantmakers. We definitely need to scale up grantmaking ability anyways.

I think that non-EAs should know that they can get funding if they do something good/useful. You shouldn't need to pretend to be an EA to get funding, and defending against people who pretend they do good projects seem easier in many cases, e.g. you can often just start giving a little funding and promise more funding later if they show they progress.

(I also expect that we/AI-risk-reduction gets even much more funding as the problem gets more known/acknowledged. I'd guess >$100B in 2030, so I don't think funding ever becomes a bottleneck, but not totally sure of course.)

I think you refer to this post. Note that there was a discussion about the title of that post, and that it was eventually changed.

In general, I think that one should be more careful about being clickbaity regarding sensitive and emotionally charged topics.

This may be counterproductive as many projects we would like to see funded face economic barriers to entry.

E.g., if starting any effective new advocacy org requires at minimum a 0.5 FTE salary of X and initial legal costs of Y, for a total of X+Y=Z, funding some people 20% below Z won't lead to a 20% less developed advocacy org, but no advocacy org at all.

Fixed costs also vary across projects, and only providing initial funding below a certain threshold could lead to certain high-value but high-fixed cost projects being de-prioritized compared to low-fixed cost, lower-value ones.

It seems like there are tools to deal with this:

• Grants ramp up and demand for talent seems stable, so most financial value is in the future. So defecting by taking the money and running, or other kinds of defection is costly.
• The incentives are even larger because grants ramp up in size and independence.
• Most grants, including community building grants, have observable outputs in early stages.
• The larger the grant is, the higher the output and more skill demanded.
• A 50K development grant for extraordinary young people has been done before. In the Thiel instance, it has been really successful, with a large number of high value projects and people.
   

Finally, this isn't what will need to happen, but an available, robust strategy is to focus on giving grants to people with high opportunity costs or high outside options. These outside options are observable and also correlated with effectiveness, for reasons most people find acceptable. 

I was considering writing something like this up a a while back, but didn't have enough evidence directly, was mostly working of too few examples as a grantmaker and general models. Glad this concern is being broadcast.

I did come up with a proposal for addressing parts of the problem over on the "Please pitch ideas to potential EA CTOs" post. If you're a software dev who wants to help build a tool which might make the vultures less able to eat at least parts of EA please read over the proposal and ping me if interested.

For what it's worth, Atlas has two different steps of online application, and then a final interview. This doesn't make it impossible to Goodhart, but it buys us time.

My prior is that one's degree of EA-alignment is pretty transparent. If there are any grifters, they would probably be found out pretty quickly and we can retract funding/cooperation from that point on. 

Also, people who are at a crossroads of either being EA-aligned or non-EA aligned (e.g., people who want to be a productive member of a lively and prestigious community) could be organizationally "captured" and become EA-aligned, if we maintain a high-trust, collaborative group environment.