With all the recent media attention on EA, I think what's really unclear to outsiders is the relationship between EA and Anthropic (rightfully so!). To most people it doesn't make sense that a community so focused on AI safety can also be linked to one of the leading companies. I'd personally be very interested to see a survey of what EAs think of Anthropic, but maybe that's a minefield.. Even a good explainer meant for the public could go a long way.
>I made a 13x13" hole in my wall
I was away for the day and when I got home, he said "I made a hole in our wall but it can be refilled."
I continue to be super impressed with RP's CCF, which is a major intellectual challenge to take on.
I've been thinking about it quite a lot since I saw it launch last year and learnt it uses the theory of moral uncertainty I co-developed, the Moral Marketplace (MMT; published in Ergo) which RP calls 'proportional allocation'. We only proposed a 'toy model' there, whereas RP has operationalised it. That prompted me to think harder about how to apply MMT in reality. I gave a talk on this at EAGxOxford last weekend to force me to do that.
I've got two main comments so far about where the model could be improved. FWIW, here are my talk slides which may not be very easy to follow as they are. I'll try and write those up into an essay in the next few weeks.
First, I think there's a more theoretically appropriate way to divide up morality than the 14 theories the CCF has. I do not think it's 'carving reality and its joints'. Moral theories are like LEGO, and built up out of the combined set of choices one makes across each debate in ethics. As such, the way that seems maximally appropriate would be having one 'bucket'/'worldview' for each possible moral theory. That ends up being way too complicated, as there are something like 6,000 distinct moral theories, and this is before one starts thinking about scalar issues. I suggest a shortcut - thinking about which moral debates are decision-relevant - but I confess I haven't fully worked it out or understood how RP's model works. But potentially the CCF's outputs are quite sensitive to how one divides up moral theories/worldviews, and there may be a better way to do this.
Second, practically - and semi-self-interestedly - another issue with the CCF is that the Happier Lives Institute's Fund doesn't feature anywhere, and isn't listed as something you're planning to look at. The maxim for MMT is that you allocate your resources to each project in proportion to your credence that it is the best. Given this, is the CCF implicitly assigning a 0% credence to (human) welfare consisting in happiness? A 0% credence that improving human happiness could be the best thing? Neither of those seem plausible, nor I assume are not what RP's staff believe. Potentially there's another explanation, but I am not sure what it would be! Hence, there seems to be at least one significant practical gap stemming from the model's design.
I'm happy to advise on further stages and, once again, congratulations with your work so far on this truly Heruclean task.
Why am I not bullish on civilizational refuges for reducing bio xrisk? I believe bio xrisk is concentrated in E2H scenarios or AI takeover, and neither of these things benefit much from refuges. For both of these threat models, there is value in refuges but only if they are a safe staging area to take back control of the environment or against an AI, which means you probably need a full industrial base worth of people/infrastructure to be protected. Society-wide biohardening and PPE is therefore the better option even on purely longtermist grounds (and has the additional property of also being much more valuable on non-longtermist grounds as well!)
Wanted to share a quick update on my comments on AI safety comms from a couple weeks ago. The NY Post is now reporting that the FTC is investigating OpenAI, Anthropic and METR, with formal demands (similar to subpoenas) reportedly coming in the next few weeks.
I think Brian Martin's Backfire framework could be useful in responding here. Martin found that power holders use five methods to suppress outrage, and there's something we can do in response to make it backfire:
I've worked on a few similar cases in the past two years, mostly in the pro-democracy space, including the FBI's arrest of Milwaukee County Judge Hannah Dugan. We built a coalition beyond the usual suspects and brought her case to the public with polling that showed a clear message to the general public. We hosted a vigil before her trial and a "Court of Public Opinion" outside the courthouse each time she appeared before a judge.
Making this backfire will be easier to do now, before the attack lands. Crisis comms is 95% preparation and relationships, 5% the crisis itself.
Would love to talk with more people about this!
*AI use: I used AI to tighten up the backfire bullets and intend to lengthen this piece into a longer one.
This comment section felt a bit brigaded here, which I don't love.
I'm sympathetic to debating strategic disagreements in the animal welfare movement, and appreciate you bringing these ideas up and pushing for what you believe is the right approach.
There were 2 things that stood out to me:
I don't think utilitarianism gives a preference to welfare reforms — it's that welfare reforms have a strong track record of being effective, while these other approaches don't. Peter Singer himself wholeheartedly embraces animal liberation, literally! If the utilitarians thought that things beyond welfare reforms would work, and were cost-effective, they'd do them. It isn't because of their principled commitment to utilitarianism that they disagree.
Secondly, I think that many of the donors and EAs would stop being EA donors or EAs at all if deontology became the norm. At least speaking for myself, I am not aligned with EA because it is EA, but I'm aligned with EA because it is more aligned with my values (which are very utilitarian). I suspect this is true for many EA-motivated people. If EA changes, it doesn't mean the donors will change their values.
I think an important question for people who aren't excited about new funding entering the space going to welfare reforms is "why are these new donors are interested in funding animal welfare work at all?"
I think the answer is because cage-free campaigns, etc. demonstrated that helping animals at scale was possible, and because welfare-motivated advocates demonstrated that something could be done to help animals effectively.
No animal welfare advocate controls where these donors give — if the movement today decided to pivot to plant-based advocacy and stop doing welfare stuff, it's plausible significantly less money would be donated to helping animals. There isn't a pot of unlimited money controlled by animal advocates — there are donors who have specific values and priorities, and organizations who will make the case to those donors that their work should be funded. The pitch should be to the donors (who actually will decide these things), not to EA as a whole, and has to overcome the fact that you're pitching against the thing that might have gotten many of them excited about animal advocacy in the first place.