Context: A slew of slightly disorganized thoughts I’ve had in mind for some time, but I’m not sure how obvious the ideas are to others. Relatively unpolished but posting anyway.
Years ago, I did a research project on value drift in EA. One of the things I asked participants is what would make them leave the movement. A couple of them mentioned the possibility of the EA movement’s values drifting.
At the time, this seemed kind of weird to me. Where could EA values drift to anyway? The only two values I could think of that define EA are effectiveness and altruism, and it seemed weird that we wouldn’t notice if one or both fell through.
As I’ve watched the movement change and grow over the past six years, I’ve realized it’s a little more complicated than that. There are lots of different ways people’s values affect how they pursue Effective Altruism. And I can’t help but notice that the values of the core of the EA movement are shifting. I’m hardly the first person to point out that things are changing, but most of what I’ve seen focuses on the increased funding. I see a few more differences:
- Longtermism — and perhaps implicitly, x-risk avoidance — as a dominant value, if not the only value. (This one’s been elaborated on at length already.)
- Less risk aversion. EA’s drastically increased involvement in policy and politics seems like the most obvious example, but the poor optics of highly-paid junior roles at “altruistic” nonprofits is an underrated risk, in my opinion, too.
- Similarly, less proportion of funding towards evidence-backed ideas, and more funding for long-shot projects, of which much or most of the expected value comes from a small chance of big success.
- Less impartiality. This is the one that concerns me the most. As more funding goes to community-building, a lot of money seems to be going to luxuries EAs don’t need, which feels very far from EA’s origins.
I’m not saying these changes are all bad, but they certainly do not reflect the values of the movement as a whole, and I want things to stay that way. But I worry that as the movement moves more and more firmly toward these values, others might be ostracized for staying where the movement was ten years, five years, or even one year ago.
My main point is: I think EAs think that leaving EA is value drift, and value drift is bad, and therefore leaving EA is a bad thing. But with the changes we’re seeing, people might be leaving EA as a means of staying within their values. And that’s okay.
By no means do I think that old EA values are superior to new ones. I’m grateful for the people challenging the status quo in EA and leading to progress. But I also want to make sure we still value the community members who don’t change their minds because of what’s trending in EA, and who still hold the values that we thought were good values 10 years ago and are likely still good values today.
Sure, here's the ELI12:
Suppose that there are two billionaires, April and Autumn. Originally they were funding AMF because they thought working on AI alignment would be 0.01% likely to work and solving alignment would be as good as saving 10 billion lives, which is an expected value of 1 million lives, lower than you could get by funding AMF.
After being in the EA community a while they switched to funding alignment research for different reasons.
Both of them assign the same expected utility to alignment-- 1 billion lives. As such they will make the same decisions. So even though April made an epistemic update and Autumn a moral update, we cannot distinguish them from behavior alone.
This extends to a general principle: actions are driven by a combination of your values and subjective probabilities, and any given action is consistent with many different combinations of utility function and probability distribution.
As a second example, suppose Bart is an investor who makes risk-averse decisions (say, invests in bonds rather than stocks). He might do this for two reasons:
These different combinations of probability and utility inform the same risk-averse behavior. In fact, probability and utility are so interchangeable that professional traders-- just about the most calibrated, rational people with regard to probability of losing money, and who are only risk-averse for reason (1) -- often model financial products as if losing money is more likely than it actually is, because it makes the math easier.