Expected value maximization hides a lot of important details.
I think a pretty underrated and forgotten part of of Rethink Priorities' CURVE sequence is the risk aversion work. I think the defenses of EV against more risk-aware models seem to often boil down to EV's simplicity. But I think that EV actually just hides a lot of important detail, including, most importantly, that if you only care about EV maximization, you might be forced to conclude that worlds where you're more likely to cause harm than not are preferable.
As an example, imagine that you're considering a choice that can cause 10 equally possible outcomes. In 6 of them, you'll create -1 utility. In 3 of them, your impact is neutral. In 1 of them, you'll create 7 utility. The EV of taking the action is (-6+0+7)/10 = 0.1. This is a positive number! Your expected value is positive, even though you have a 60% chance of causing harm. In expectation you're more likely than not to cause harm, but also in expectation you should expect to increase utility a bit. This is weird.
Scenario 1
More concretely, if I consider the following choices, which are equivalent from an EV perspective:
Option A. A 0% chance of causing a harmful outcome, but in expectation will cause +10 utility
Option B. A 20% chance of causing a harmful outcome, but in expectation will cause +10 utility
It seems really bizarre to not prefer Option A. But if I prefer Option A, I'm just accepting risk aversion to at least some extent. But what if the numbers slip a little more?
Scenario 2
Option A. A 0% chance of causing a harmful outcome, but in expectation will cause +9.9999 utility
Option B. A 20% chance of causing a harmful outcome, but in expectation will cause +10 utility
Do I really want to take a 20% chance on causing harm in exchange for 0.001% gain in utility caused?
Scenario 3
Option A. A 0% chance of causing a harmful outcome, but in expectation will cause +5 utility
Option B. A 99.99999% chance of causing a harmful outcome, but in expectation will cause +10 utility
Do I really want to be exceedingly likely to cause harm, in exchange for a 100% gain in utility?
I don't know the answers to the above scenarios, but I think it feels like just saying "the EV is X" without reference to the downside risk misses a massive part of the picture. It seems much better to say "the expected range of outcomes are a 20% chance of really bad stuff happening, a 70% chance of nothing happening and a 10% of a really really great outcome, which all averages out to an >0 average". This is meaningfully different than saying "no downside risk, and a 10% chance of a pretty good outcome, so >0 average".
I think that risk aversion is pretty important, but even if it isn't incorporated into people's thinking at all, it really doesn't feel like EV produces a number I can take at face value, and that makes me feel like EV isn't actually that simple.
The place where I currently see this happening the most is naive expected value maximization in reasoning about animal welfare — I feel like I've seen an uptick in "I think there is a 52% chance these animals live net negative lives, so we should do major irreversible things to reduce their population". But it's pretty easy to imagine doing those things being harmful, or your efforts backfiring, etc. in ways that cause harm.
Reflections on a decade of trying to have an impact
Next month (September 2024) is my 10th anniversary of formally engaging with EA. This date marks 10 years since I first reached out to the Foundational Research Institute about volunteering, at least as far as I can tell from my emails.
Prior to that, I probably had read a fair amount of Peter Singer, Brian Tomasik, and David Pearce, who might all have been considered connected to EA, but I hadn’t actually actively tried engaging with the community. I’d been engaged with the effective animal advocacy community for several years prior, and I think I’d volunteered for The Humane League some, and had seen some of The Humane League Labs’ content online. I’m not sure if The Humane League counted as being “EA” at the time (this was a year before OpenPhil made its first animal welfare grants).
This post is me roughly trying to guess at my impact since then, and reflections on how I’ve changed as a person, both on my own and in response to EA. It’s got a lot of broad reflections about how my feelings about EA have changed. It isn’t particularly rigorously or transparently reasoned — it’s more of a reflection exercise for myself than anything else. I’m mainly trying to look at what I’ve worked on with a really critical eye. I make a lot of claims here that I don't provide evidence for.
I’m sharing this because I think the major update I’ve had from doing this is that while I’ve generally done many of the “working-in-EA” things that are often presented as high impact, I personally feel much more tangibly the impact of my donations, and right now, if I think what’s made me feel best about being in EA, it’s actions more in the earning-to-give direction than the direct work direction.
My high-level view of my impact over this period is something like:
Background
I became pretty convinced that factory farming was a moral tragedy as a little kid, I believe due to exposure to either PETA content or PETA Kids content. My brother was also vegetarian, which was a compelling enough reason for me to also become vegetarian. I volunteered for a lot of animal welfare organizations, especially in college. I also did a lot of direct action-type advocacy for animals in college. I was already a fairly hardcore utilitarian at that point, and had mainlined Peter Singer, David Pearce, Timothy Sprigge, and a bunch of other wacky utilitarians. I spent a significant amount of my time in college staying up until the early morning talking about wild animal suffering and other animal issues with my closest friend while playing the video game Super Monkey Ball. This did not help any animals but was incredibly important to how I think about animal issues now.
At some point around 2011 or 2012, I saw a frog that was hit by a bike and dying, and was really distraught over it. I’m not sure why, but this was oddly transformative for me, and I just internalized animal suffering from it really directly in a way I hadn’t before. I also have a fairly strong memory from when I was 20 or 21 of spending an afternoon in the rain putting worms back from the sidewalk into the grass, and feeling bad about them dying naturally. I formed fairly strong views about animals in nature living awful lives, and beliefs about my obligations to help them.
In 2014, I was targeted by a Google Ad for The Foundational Research Institute, I believe on a topic related to wild animal welfare. I think this was my first exposure to EA formally, though I’d read studies on The Humane League Labs website, had read Animal Liberation, Famine, Influence, and Morality, and some other books that informed EA ideas.
I did some volunteering for FRI, read a lot of Brian Tomasik’s website, and also did some experiments at a cat shelter on reducing the impact of outdoor cats on animals. In 2016, I started working at Mercy For Animals, running corporate animal welfare campaigns. I also formally started Utility Farm, a nonprofit that would later merge with Wild-Animal Suffering Research into Wild Animal Initiative. I’ve done a bunch of other things in the EA world since.
My potential impact
My beliefs in 2014 compared to now
This is my best effort to estimate how my credence in various beliefs have changed since 2014, based on notes and exercises from that period of my life.
2014
2024
Change
90%
90%
+0%
95%
60%
-35%
85%
20%
-65%
75%
10%
-65%
85%
15%
-70%
85%
80%
-5%
75%
65%
-10%
75%
80%
+5%
30%
40%
+10%
5%
10%
+5%
85%
35%
-50%
85%
60%
-25%
95%
10%
-85%
90%
15%
-75%
90%
30%
-60%
60%
85%
+25%
80%
95%
+15%
How my thinking about EA has changed over time
I have some long-held views that haven’t really changed:
My views have also changed in a bunch of ways:
Things that changed about me from exposure to EA
I care a lot more about money
I care a lot more about status
My commitment to doing good feels deeper
I feel more morally compromised
Overall, when I look at my first 10 years engaging with EA, I feel mainly like things are just ambiguous to me. I feel a lot more positive about some donations I made than anything else — in particular large donations to brand new projects that probably helped accelerate them a lot. The animal work I’ve done feels promising but ambiguous. This post feels very melancholy, but ultimately, I still feel excited about trying to do impactful work in the world.
Nothing super constructive to say, but two years later, I still think about this post a lot and keep coming back to it to read some specific bullets. It's somewhat moving, but also very content-dense, one of my favorite pieces of writing/information on here.
I generally find pieces about how people's views and drives evolve helpful on many levels, and I'd welcome more of them, even from much less senior people. I try to keep track of this for myself: it's useful for gaining insights into, for example, what actually influences my priorities over time and how contingent those drives are.
This is really great and I'd encourage you to convert it to a full post! It's absolutely worthy of that honor :)
Thank you for writing this! Was really interesting to read. I'd love to see more posts of this nature. And it seems like you've done a lot for the world — thank you.
I have a couple questions, if you don't mind:
You write
I would love to hear your reasoning (pessimism about fulfillment? WAW looking better?) and what sort of evidence has convinced you. I think this is really important, and I haven't seen an argument for this publicly anywhere. Ditto about your skepticism of the organizations leading this work.
Did you mean to change one of the years in the two statements of this form?
I'd love to hear more about this. How much value do you think e.g. the median EA doing direct work is creating? Or, put another way, how significant an annual donation would exceed the value of a talented EA doing direct work instead?
Thanks for the questions!
Corporate campaigns
I would love to hear you talk more about this :) What makes you hopeful that scalable interventions are coming, and can you say more about anything you're particularly excited about here? Also curious what "aren't that far away" caches out into in terms of your beliefs -- in 1 year? 3?
I wonder if your opinions are related to the following, which I'd also be excited to hear more about!
(Thanks for sharing this post Abraham, I enjoyed reading it :) )
Thanks for the questions!!
The ones that seem most likely in the near future are:
Things that make me think this is on the table:
In terms of timelines — I think this is more like 10-15 years. But part of the reason I think that's exciting is that I used to think it would be more like 2050+ before anything like this was on the table. I think I've also just generally decreased my confidence that the problems as are as difficult as I thought before (though I definitely think they are still tricky).
For insecticides, I think my view remains that we are something like 2-5 years of specific lab/field research away from plausibly having a great intervention, so it is sad that progress hasn't been made on it, and given that this also seemed like the case a few years ago, funding the research should have been a priority earlier.
Thanks for writing this, heaps of interesting points. Most surprising and saddening to me was that you think there is a 70% chance EA will be net-negative! Could you explain why you think this? Your various concerns about power centralisation and so forth make sense to me, but to my mind this isn't nearly enough to flip the sign, and EA still seems overwhelmingly good to me.
I was also struck by your melancholy tone - somehow I think I implicitly hoped that if I accomplished all the things you have I would feel more resoundingly happy with my impact! But maybe EAish people are unusually cognisant of missed opportunities and impact that could have been but wasn't.
I don't think it's all net-negative — I think there are lots of worlds where EA has lots of good and bad that kind of wash out, or where the overall sign is pretty ambiguous in the longrun.
Here are lots of ways I think are possible EA could end up causing a lot of possible harm. I don't really think any of these are that likely on their own — I just think it's generally easier to cause harm than produce good, so there are lots of ways EA can accidentally not achieve being overall positive, and I generally think it has an uphill road to climb to end up not being a neutral or ambiguous quirk in the ash heap of history.
I note that these risks hardly apply to GHD work ;).
Can you explain how FTX harm could plausible outweigh good done by EA? I can't fathom a scenario where this is the case myself.
Yeah, I think there are probably parts of EA that will look robustly good in the long run, and part of the reason I think that it's less likely EA as a whole will be less likely to be positive (and more likely to be neutral or negative) are that actions in other areas of EA could impact those areas negatively. Though this could cut both in favor of or against GHD work. I think just having a positive impact is quite hard, even more so when doing a bunch of uncorrelated things when some of them have major downside risks.
I think it is pretty unlikely that FTX harm outweighs good done by EA on its own, but it seems easy enough to imagine that conditional on EA's net benefit being barely above neutral (which for other reasons mentioned above seems pretty possible to me, along with EA increasingly working on GCRs which directly increases the likelihood EA work ends up being net-negative or neutral, even if in expectation that shift is positive value), that the scale of the stress / financial harm caused by EA via FTX, outweighs that remaining benefit. And then there is brand damage to effective giving, etc.
But yeah, I agree that my original statement above seems a lot less likely than FTX just contributing to an overall portfolio of harm or work that doesn't matter in the longrun from EA.
Really interesting, thanks for sharing. I was particularly surprised about your changes of mind here:
E.g. some spontaneous potential cruxes that might be interesting to hear your disagreement with, in case they capture your reasons for pessimism:
Nice, these are good questions, but probably don't capture all the cruxes in my view.
1. I think this seems moderately unlikely to me? I'm not sure what would drive down prices further than where they are now, as it seems like a large portion of the cost are the proteins themselves, and not production.
2. This also seems like it relies on crossing technological hurdles that are really hard.
3. I think this seems possible? But I'd put below 50% on it, and if it does happen, I'd expect something more like the climate movement, where lots of people think it is important but don't really take substantial steps to act on it.
4. I think that reaching 20% vegetarian seems possible in some countries, but I think I'm a lot more skeptical it'll go much higher.
I think it does seem plausible to me that there would be a meaningful reduction in the amount of meat consumed over this period in developed countries, but also expect that might come with more chicken/fish consumption that would offset animal welfare gains anyway.
I think another crux more important to my pessimism is that I don't feel very convinced that price/taste competitive meat alternatives will cause a significant increase in their adoption.
Chicken: Approximately 6.6 cents per gram of protein
Lentils: Approximately 3.7 cents per gram of protein
Finally, I'm also probably more optimistic about your last point, thinking that price/taste competitive meat alternatives will be huge. I think the Beyond and Impossible "moments" were huge milestones, and a few more "moments" like that will reduce resistance against higher welfare standards & higher prices for conventional meat.
Nice, these are great points.
On some specifics:
Thanks, that all makes sense and moderates my optimism a bit, and it feels like we roughly exhausted the depth of my thinking. Sigh... anyways, I'm really thankful and maybe also optimistic for the work that dedicated and strategically thinking people like you have been and will be doing for animals.
Thank you for sharing your reflections. As I read it I found various aspects that resonated with me, and I suspect that many other people on the EA Forum will feel the same. I'd love to see more of this type of writing (contemplative, reflective, critical/skeptical while being kind) on this forum.
After some discussions with someone offline that were clarifying, I want to clarify my decrease in confidence in the statement, "Farmed vertebrate welfare should be an EA focus".
I think my view is slightly more complicated than this implies. I think that given that OpenPhil and non-EA donors are basically able to fund what seem like the entirety of the good opportunities in this space, I don't think these groups are that talent constrained, and it seems like the best bets (e.g. corporate campaigns) will continue to have decreasing cost-effectiveness, new animal-focused talent should probably be mostly going into earning-to-give for invertebrates/WAW, and that donations should mostly go to groups there or the EA AWF (which should in turn mostly fund invertebrates and WAW). I don't think farmed vertebrate welfare should be the default way that EAs recommend to help animals
This an incredible set of accomplishments. Thanks for your dedication!
Thanks, Abraham, I liked reading this! Good luck for an impactful decade to come
Hello !
Thanks a lot from sharing all this knowledge. It is pretty insightfull, even for people who don't follow EA news for years.
There are several claims that surprised me a little bit. I would be pleased to have more infos about these particular claims:
1-Low probability: People generally have a deep ethical obligation to change their diet to help animals: If it is pretty clear that it is not the most efficient way to help animals, it is not that clear that we do not have a moral duty to at least not eat animal products.To my mind, I think that we have to differenciate moral duties to efficacities issues.
Moreover, it is also close to impossible to convince people that animal welfare is a problem while eating animal products
2-Low probability: We can make meaningful progress on abolishing factory farming or improving farmed animal welfare by 2050: I would be pleased to know more about the facts that leed you into thinking that.
3-Low probability: I have a strong moral obligation to prevent future negative lives from occurring: Same than two.
4- High probability: It was good for animal welfare that the EAs “won” the abolitionist/welfarist debates: I would be interested about details of the historical fact (how EA "won" that debate") and also why it is a good news.
I know it's a lot of questions. Feel free to answer from all to none :) .
Like others I also feel like you had more impact that you aknowledge to yourself :).
Thanks again for the quality of the reasoning.
Sorry to just see this!
Great reflections, Abraham!
Say you could either hire the best candidate for a role or the 2nd best plus receiving X $/year. What is the value of X which would make you indifferent between the 2 options? Feel free to provide different answers for different roles / sets of roles and organisatons/areas.
I think it would be pretty hard for me to make that trade off in a workplace context (I think I'm still a deep sucker for impact and in any real version of this is X would be whatever the organization is indifferent towards and I'd donate it). If you forced me to in some hypothetical I'd guess X is quite low for many junior roles (<$10k), but higher for more mid/senior roles (>$50k?). But I think something like the following are true:
Out of curiosity, what would you be doing?
(My guess: running an insect welfare org, or starting another EA charity.)
Yeah, I think that's basically what I was thinking (specifically, starting an insecticide charity, or similar project focused on implementing a WAW intervention)
Have you checked with potential donors if they'd be willing to pay you at a rate you find acceptable to run such a charity?
I'd be pretty excited about improving insecticides, but I'm not sure about donating much myself in the near term, since I already feel overinvested in invertebrates recently.
Also, adding to this, potential donors might be willing to pay more for you, given your experience, but maybe you've accounted for this in "market rates, etc". Presumably this would increase the probability of success of the org, from their POV.
And even bumping up the costs of the whole org 2x through higher salaries still leaves an insecticide charity at least 1/2 as cost-effective as something extraordinarily cost-effective (the same org where the same people work for less), which is still extraordinarily cost-effective!
If the counterfactual is that such a charity isn't started at all, that could be much worse than you running it at higher pay.
Great post! I'm interested in how you are 80% confident that "Most experiences are negative / suffering dominates in the wild". I can understand why you would lean towards it being negative, but why so confident given how little we still understand the experience of animals?
Yeah, I think this just seems pretty likely to me due to thinking that most animals are juveniles / die as juveniles, and the amount of time an animal has to be alive to accumulate good experiences to outweigh a painful death is probably higher than this. Things that have made me slightly less certain about this are me thinking it is more likely than I used to that adult animals in the wild live good lives, and me thinking that it is less likely than I used to that insects/some other invertebrates experience suffering, especially juvenile insects (though I probably still put a higher credence in this than many people).
I think it is pretty plausible I'm overconfident here though.
But, I also think this belief is mostly irrelevant to EAs / wild animal welfare advocates, unless you think there are special reasons improving welfare is easier on one side of the spectrum than the other, which I don't really have strongly held opinions on.
The juvenile animal argument is interesting, as from a total "QALY" perspective, if animals die very young then unless their deaths are extremely suffering-ful and drawn out, the total time for suffering isn't that large IMO.
Yep I completely agree that the belief is (or should be) mostly irrelevent to wild animal welfare advocates, and I think WAW might be more palatable to more people if it was emphasised less. "We have cheap and effective ways of helping wild animals live way better lives" is a better markteing tool than "Wild animals have bad live and are suffering soooo much so we have to do something" (aware I'm strawmanning for emphais a bit here). It only becomes relevant for arguments that look at whether the whole world is "net positive or negative", which I find a bit unhelpful as that discussion doesn't get us closer to making things better.
On that I appreciated these points
"On WAW specifically, my view is something like:
Yeah, I agree with everything you say here RE WAW, on both how to present it and the usefulness of the net-positive or negative debate.
What do you wish you'd done differently and are there any lessons for AI governance which may be in a similar stage?
I'm pretty uncertain, but I think my best guess is that starting a group/getting someone to start a group working directly on it at the time would have been better than lobbying people to care about it. I suspect that broadly applies.