Reflections on Anthropic and EA

"I think EA’s failure to grapple with the corrupting influence of power is among its greatest failures. "

This has been the feature of forum discussions that has disturbed me possibly the most since joining. People don't like to put any weight on conflicts of interest even when the person arguing a point has a huge amount to gain. "Just argue the object point" people say, don't bring up the conflict of interest... 

People seem surprised and bewildered when AI folks defect away from AI safety towards capabilities. People trust that as AI companies grow, those gaining power and money from shares will not be adversely influenced by that power and money.

Even as I have gained a teeny weeny bit of power just in a teeny weeny corner of the global health world, I have felt a little of the corrupting influence. Living far away from this in Uganda, I'm not part of this at all and like you it's very unclear what can be done to help, but talking about it a bit could be better than nothing. I loved this post thank you!

This is a beautiful piece Abraham, thanks for writing it. I feel very similarly to you. I thought this EA Forum post from the FTX-era hit a few of these concerns well, as well as this comment from Benjamin Todd:

One way to see the problem is that in the past we used frugality as a hard-to-fake signal of altruism, but that signal no longer works.

Basically, it feels harder to know who is genuine and who to trust vs who is involved for the various status-based and financial incentives. Whilst not new, I feel like I've seen an increasing number of organisations/individuals who are functionally cosplaying being interested in EA, to increase their chances of getting funding. This makes me sad - I would love not to have to question people's motives like that, but it feels necessary sometimes.

Also, the demanding part of EA is something I really value too (in fact, I wrote a relatively controversial post on some issues with paying high salaries in EA orgs shortly before the FTX crash). On the frugality aspects of demandingness: I feel torn on how to navigate this. As I say in the post above, I worry about losing some ideological commitment (and related impact-focused decision-making) by paying generous salaries and attracting new people. But at the same time, I am very happy that we can pay more as a movement, if it means attracting great people. Similarly, even though people can often fairly justify spending significant chunks of money to increase their productivity, this kind of thinking still makes me uneasy sometimes (the most obvious example being a $2k coffee table).

The most charitable explanation of the tension here is that people just disagree with you about what is most impactful. I appreciate your transparency in considering whether aesthetics and nostalgia for a previous era of EA might be driving your unease. 

Ultimately, it is better to debate the merits of specific interventions than general vibes. I think even the Anthropic folks would agree that, e.g., moving to SF is purely instrumental to some more specific theory of change that may or may not have merit. 

During the FTX era, it lost that for a while. Getting your project funded basically required that you had a pulse and sounded like an EA.

Do you mean just in AI safety/meta? Because the FTXFF was only funding long-term projects, so I know there were still huge funding gaps in global poverty and animal welfare. And even in bio and nuclear security, there were still very large funding gaps.

I have been offered jobs that are billed as impactful and that pay vastly more money than I need while not providing any clear benefit for the world.

That's great that you are so successful. But there are many EAs/rationalists who are well-qualified who still have not gotten an EA job (probably thousands). I think part of the disconnect between the people saying there is lots of money and the people who are saying that they can't get a job is that EAs are so well-qualified that they are used to only having to apply for a few jobs per offer. The number of applicants per job is something like 30-300, so it makes sense that the average person needs to apply to that many to get a job. But when EAs apply to 10 to 30 EA jobs and don't get one, they tend to get frustrated. Also there are just not that many EA jobs that would be relevant to an individual EA's qualifications.

Furthermore, there are enormous potential effective uses of money that don't require EA labor, such as Give Directly, stockpiling PPE, etc.

So overall, I do think we should be careful to limit grift, but there is enormous room for effective use of funding in EA, one estimate was ~$1 trillion.

Immense, undemocratic political spending is ramping up (though spending by the enemies of AI safety is also growing)

You present the parenthetical as a meliorating factor, but I expect that these enemies exist due to previous undemocratic power-seeking actions by the AI safety community.

(This isn't based on any private information, I just think there must be some reason these enemies single out EAs in particular. Bad faith actors don't just randomly pick targets to attack. My best guess at the reason is the intense focus on gaining influence and power.)

This is, of course, very tempting, and sounds very exciting. It would be a glorious coincidence if, of all possible things I could be doing, the thing that is most intellectually interesting and fun and increases my status the most also happens to be the most impactful thing I could do with my life. Similarly, why is it that taking a high-paying, high-power job where the promise of impact is a vague sense of “influence” at the cost of progress on a difficult, high-risk, and concrete intervention is so appealing?

Reminds me of this classic: Beware surprising and suspicious convergence.

Thanks for writing this.

I started my EA journy at Giving What We Can, encouraging others to pledge to give 10% of their income to effective charities. I got into allt his because I thought it was basically insane that you could save a life for $5k. I still do, but now I'm working in AI, where there's significant amounts of funding available, powerful people involved and interested, the stakes are existential, I feel like my scope sensitivty cannot keep up anymore. So lot of what you're saying resonates.

I think EA’s failure to grapple with the corrupting influence of power is among its greatest failures.

I both agree with this in the abstract and also feel pretty clueless about what grappling with this would look like. I worry that one way to grapple with it is to try to avoid it and flee to a place with less moral ambiguity, but also at the cost impact. I'd be interested in your thoughts on what it would look like for people who are wielding that power to take seriously its corrupting influence.

Thanks for writing this. As someone who has at times steered people toward political donations, I wanted to chime in and say: I'm scared, too.

Maybe I'm living under a rock but I don't see any concrete evidence of these claims of people saying the jobs that are paying ridiculous amounts of money have 10-1000x impact.

I think EA struggles to deal with power at all, even beyond its corrupting or enabling capabilities. There really isn't an analytical framework in EA for thinking about the way emerging institutional environments generate their own value hierarchies and reward systems. They are often ones that people experience as natural but that are in fact products of their group's structure. This can even includes the very advice you describe: the exhortations to move to SF, or to redirect your charity work toward a more impactful area, whatever that may be. I'm sure the advice is genuine and shared in good faith. And even feels entirely accurate from inside. But from the outside, it reads, rightly, as conditional. I'm really glad you shared this, and I hope it forces the right people to ask: despite how we explicitly founded and built the EA movement, how do we resist the corruption of our goals now that there will be large and sustained infusions of cash? I also thought your note about standards was precient. The amount of funding coming will also attract grifters, which requires also asking what community hygiene and defensive measures will be necessary to reduce their ability to derail EA goals. 

Many people will compromise their morals for money. That's life. I try not to hold it against them.

For what it's worth, EA's donor core of pledgers/EtGers probably aren't going anywhere or compromising anything much, being a group of people who constantly could just decide to have more money and don't. So maybe they'd be a nicer group for you to hang out with if this kind of moral compromise really bugs you?

Personally I'd quite like to hear about your new charity.

I think quite a few dedicated EAs will remain whatever happens. Perhaps those voices actually have more leverage in another way: To see if there are ways to redirect money away from high-paying less impactful jobs and into animal welfare and/or global health/poverty reduction. I have always been suspicious of the extreme assumptions needed behind x-risk interventions being prioritized over other causes: Extreme increases in value (populations and life satisfaction) over long time horizons, as well as a simultaneous ceasing "forever" of subsequent periods of high risk. In a way, it becomes EA all over again. As less clearly useful AI safety project pop up left and right, that might look more like philanthropy before EA. And our job is then to show them that there are more effective interventions than AI safety. Renaissance Philanthropy is doing fantastic work looking at how new technology can be applied to solve current, painful problems - something making me hopeful.

Worth mentioning because the policy is so new: your disclosure was interesting but isn't required. Disclosure is only required when your post contains significant LLM-generated text, so you're all good, and can cut it unless you included it by choice. 

"LLM disclosure: I wrote this post myself, then asked an LLM to copy-edit it before posting. I manually made any edits I liked and copy-pasted no text from the LLM (my current practice for using LLMs in writing that I care about)."

You worry about Anthropic's money being a corrupting influence, but their whole company is far far worse than FTX, because of the existential risk it's subjecting the entire world to. Not only are they continuing in the suicide-onmicide race, they are now leading it! 

If you work at Anthropic, please quit and make a big public statement about this. You shouldn't be waiting for the IPO, because there's not much point in having millions to donate if we're all dead in short order (because of how those millions were made). If you own shares, sell them (even pre-IPO, if you can). I no longer consider myself EA because of this issue. Whilst the money and power in EA is pro-Anthropic, I don't want to associate with it.

Beautiful and moving post.

It reminded me of very good post by Ben Kuhn about the risk of grifters - https://forum.effectivealtruism.org/posts/nqt3kasPieQEKihCp/the-biggest-risk-of-free-spending-ea-is-not-optics-or

Right now, I’m leaving my job to go start a new charity, working on a very narrow intervention.

What is it? I'm curious!

https://forum.effectivealtruism.org/posts/48mypEepqBqWibKtJ/?commentId=WQfiqqSpsjYBGqtKC

I wrote a related parable in response to a similar-ish post a while ago if people like fiction.