The case of the missing cause prioritisation research

by weeatquince16 min read16th Aug 202084 comments

228

Criticism (EA Movement)Movement StrategyForum PrizeCause Prioritization
Frontpage

Introduction / summary 

In 2011 I came across Giving What We Can, which shortly blossomed into effective altruism. Call me a geek if you like but I found it exciting, like really exciting. Here were people thinking super carefully about the most effective ways to have an impact, to create change, to build a better world. Suddenly a boundless opportunity to do vast amounts of good opened up before my eyes. I had only just got involved and by giving to fund bednets and had already magnified my impact on the world 100 times. 

And this was just the beginning. Obviously bednets were not the most effective charitable intervention, they were just the most effective we had found to date – with just a tiny amount of research. Imagine what topic could be explored next: the long run effects of interventions, economic growth, political change, geopolitics, conflict studies, etc. We could work out how to compare charities of vastly different cause areas, or how to do good beyond donations (some people were already starting to talk about career choices). Some people said we should care about animals (or AI risk), I didn’t buy it (back then), but imagine, we could work out what different value sets lead to different causes and the best charities for each.

As far as I could tell the whole field of optimising for impact seemed vastly under-explored. This wasn’t too surprising – most people don’t seem to care that much about doing charitable giving well and anyway it was only just coming to light how truly bad our intuitions were at making charitable choices (with the early 2000’s aid skepticism movement).

Looking back, I was optimistic. Yet in some regards my optimism was well-placed. In terms of spreading ideas, my small group of geeky uni friends went on to create something remarkable, to shift £m if not £bn of donations to better causes, to help 1000s maybe 100,000s of people make better career decisions. I am no longer surprised if a colleague, tinder date or complete stranger has heard of effective altruism (EA) or gives money to AMF (a bednet charity).

However, in terms of the research I was so excited about, of developing the field of how to do good, there has been minimal progress. After nearly a decade, bednets and AI research still seem to be at the top of everyone’s Christmas donations wish list. I think I assumed that someone had got this covered, that GPI or FHI or whoever will have answers, or at least progress on cause research sometime soon. But last month, whilst trying to review my career, I decided to look into this topic, and, oh boy, there just appears to be a massive gaping hole. I really don’t think it is happening.

I don’t particularly want to shift my career to do cause prioritisation research right now. So I am writing this piece in the hope that I can either have you, my dear reader, persuade me this work is not of utmost importance, or have me persuade you to do this work (so I don’t have to).

 

 

A. The importance of cause prioritisation research

What is your view on the effective altruism community and what it has achieved? What is the single most important idea to come out of the community? Feel free to take a moment to reflect. (Answers on a postcard, or comment).

It seems to me (predictably given the introduction) that far and away the most valuable thing EA has done is the development of and promotion of cause prioritisation as a concept. This idea seems (shockingly and unfortunately) unique to EA.[1] It underpins all EA thinking, guides where EA aligned foundations give and leads to people seriously considering novel causes such as animal welfare or longtermism.

This post mostly focuses on the current progress of and neglectedness of this work over the past few years. But let us start with a quick recap of why cause prioritisation research might be important and tractible. The argument is nicely set out in Paul Christiano’s The Case for Cause Prioritization as the Best Cause (written 2013-14). To give a short summary Paul says:

1. Some causes are significantly higher impact than others. We theoretically expect and empirically observe impact to be “heavy tailed” with some causes being orders of magnitude more impactful (see also Prospecting for Gold). We should not yet be confident in our top causes and many of our current approaches to improve the world rely on highly speculative assumptions (eg about long term effects). So if we could make progress on prioritisation we should expect to have a large positive impact. 

2. it is reasonable to think that research would make progress because:

  • Very little research has been done on this so far.
  • The work that has been done suggests that progress is difficult but not impossible.
  • We can see research programs that could be useful (see some of my ideas below).
  • Human history reflects positively on our ability to build a collective understanding of a difficult subject and eventually make headway.
  • Even if difficult, we should at least try! We would learn why such research is hard and should keep going until we reach a point of diminishing returns.

(Also this week 80000 Hours has just written this: Why global priorities research is even more important than I thought)

In short:

Cause prioritisation is hugely valuable to guide how we do good.

 

 

B. The case of the missing cause prioritisation research

Let me take you through my story, and set out some of the research gaps as I have experienced them.

 

Community building

From 2013 until 2017 I ran the EA community in London. I set myself the goal of building a vibrant welcoming and cohesive community and I like to think I did OK. But occasionally the intellectual framework was just not there. For while I might say “we are a new community, we don’t yet have the answer to this” but after a few years the excuse got thin. The research on specific causes areas got deeper, but the cause prioritisation research did not. In particular I struggled to provide materials to people who did not fall close to thinking along classical utilitarian lines.[2]

And it was damaging. It is damaging. More and more, as I look across the EA movement I see the people who join are not those who are open minded souls keen to understand what it means to do the most good, but people who are already focused on the causes we champion: global development or animal welfare or preventing extinction risk. Now I love my cause committed compatriots, but I do think we are at risk of creating a community that is unwelcoming to the true explorers, a community that is intellectually entrenched and forever doomed to only see those three cause areas.

I think we need to do cause prioritisation from the point of view of different value sets and different cultures. This is important for building a good community, especially for spreading to other countries (as discussed here and here). This is also important for reaching truth. Different people with different life experiences will not only ask different questions, but have different hypotheses about what the answers might be.[3] 

I could say more on this but honestly I think most of it is covered in the amazing post by Objections to value alignment between EAs by CarlaZoeC which I recommend you check out.

 

Parliament

One thing I notice is that, with few exceptions, the path to change for EA folk who want to improve the long-run future is research. They work at research institutions, design AI systems, fund research, support research. Those that do not do research seem to be trying to accumulate power or wealth or CV points in the vague hope that at some point the researchers will know what needs doing.

Post community building I moved back into policy and most recently have found myself in the policy space, building support for future generations in the UK Parliament. Not research. Not waiting. But creating change.

From this vantage point it doesn’t feel like the EA community has thought much about policy. For example there is a huge focus on AI policy, but the justification for this is weak. Even if you fully believe the longtermist arguments that top programmers should work on AI alignment, it does not immediately follow that good policy people can have more long term impact in AI policy compared to policy on resilience, macroeconomics, institution design, nuclear non-proliferation, climate change, democracy promotion, political polarisation, etc, etc.

Most of the cause prioritisation research has been focused on how to do good with money. But there is very little on how to do good if you have political capital, public status, media influence and so on. Trying to weigh up and compare all the different policy approaches I list above would be a mighty undertaking and I do not expect answers soon, but it would be nice to see someone trying to take on the task, and not focusing solely on where to shift money. 

 

My own values

Most recently I have been thinking about what career route to go down next, what my values are, and what has been written on cause prioritisation.

Looking around it feels a like there is a split down the middle of the EA community:[4] 

  1. On the one hand you have the empiricals: those who believe that doing good is difficult, common sense leads you astray and to create change we need hard data, ideally at least a few RCTs.
  2. On the other side are the theorists: those who believe you just need to think really hard and to choose a cause we need expected value calculations and it matters not if calculations are highly uncertain if the numbers tend to infinity.

Personally I find myself somewhat drawn to the uncharted middle ground. Call me indecisive if you like but it appears to me that both ends of this spectrum are making errors in judgement. Certainly neither of the approaches above come close to how well-run government institutions or large successful corporations make decisions.

(I also don’t think these two areas are as far apart as it first seems. If you look at the structural change and policy research GiveWell is interested in it is not too far away from long-termist research suggestions on institutional change.)

I think this split provides a way of breaking down the work I would love to see:

 

Beyond RCTs – It would be lovely to see the ‘empiricals’ crew move beyond basic global health, to have them say “great we have shown that you can, despite the challenges, identify interventions that work and compare them. Now let’s get a bit more complicated and do some more research and find other interventions and consider long run effects and so on”. There could be research looking for strong empirical evidence into:

  • the second order or long run effects of existing interventions.
  • how to drive economic growth, policy change, structural changes, and so forth.
  • unexplored areas that could be highly impactful such as access to painkillers or mental health. (There could be experimental hits based giving.)

It honestly shocks me that the EA community has had so little progress in this space in a decade.

 

Beyond speculation – it would be great if the ‘theorists’ looked a bit more at making their claims more credible. From my point of view, I could save a human life for ~£3000. I don’t want to let kids die needlessly if I can stop it. I personally think that the future is really important but before I drop the ball on all the things I know will have an impact it would be nice to have:

  • Some evidence that we can reliably affect the future: What empirical evidence is there that we can reliably impact the long run trajectory of humanity and how have similar efforts gone in the past?
  • Cause and intervention prioritization. What are the options, the causes and interventions to influence the long-term, which of these can be practically impacted, have feedback loops that can be used for judging success, and so forth? I would love to see more comparisons of causes like improving institutions, increasing economic growth, global conflict prevention, etc.
  • Less dodgy reasoning. I am not going into here all the errors, groupthink, and mistakes that I think EA longtermists often make. Let me give just one example, if you look at best practice in risk assessment methodologies[5] it looks very different from the naive expected value calculations used in EA – if someone tells me to dedicate my life to stopping global risks it would be good if I was confident they actually understood risk mitigation. I think there needs to be much better research into how to make complex decisions despite high uncertainty. There is a whole field of decision making under deep uncertainty (or knightian uncertainty) used in policy design, military decision making and climate science but rarely discussed in EA.

 

In short:

You could categorise this research in a bunch of different ways but if I had to make a list the projects I would be super excited to see are:

  1. The basics: I think we could see progress just by doing investigations of a broad range of different potentially top causes and comparisons across causes. (The search for “cause X”).
  2. Consideration of different views and ethics and how this affects what causes might be most important.
  3. Consideration of how to prioritise depending on the type of power you have, be it money or political power or media influence or something else.
  4. Empirical cause selection beyond RCTs. The impact of system change and policy change in international development and more consideration of second order effects.
  5. Theoretical cause selection beyond speculation. Evidence of how to reason well despite uncertainty and more comparisons of different causes.

This research would ensure that we continue to learn how to do good, not entrenched in our ways, and taking the actions that will have the biggest impact on the world.

 

 

C. Whodunnit?

So is anyone doing this? Lets run through my list. 

[Edit: disclaimer, I have looked though organisations plans, research agendas and so forth and done the best I can but I did not invest time in talking to people at all the organisations in this space – so it is possible I may have mischaracterised specific organisations compared to how they would describe themselves – apologies]

 

1. The basics – partially happening – 5/10

Shallow investigations of how to do good within a few cause areas are being done by Open Philanthropy Project (OpenPhil) and to a lesser extent by Founders Pledge (FP). The main missing part is that there is little written that compares across these different causes or looks at how one might prioritise one cause over another (except for occasional mentions in the FP reports and the OpenPhil spreadsheets here and here). 

More granular, but still high level intervention research is being done by Charity Entrepreneurship.

 

2. Different views – not happening – 0/10

No organisation is doing this. There is no systematic work in this space. The most that is going on is a few individuals or small groups that have taken up specific approaches (still largely hedonistic utilitarianism adjacent) and run with it, such as the Happier Lives Institute (HLI) or the Organisation for the Prevention of Intense Suffering (OPIS).

 

3. Policy and beyond – not happening – 2/10

No organisation is doing research into how to prioritise if you have political power or media influence or something other than money. 80000 Hours (80K) appeared to do some of this in the past but are now focusing on their priority paths. They have said that the details of what those paths are may change. It is unclear if such changes indicate that they will do more research themselves or if they expect to change in light of others research. Either way  the rough direction feels fairly set so I do not expect much more high level cause prioritisation research from them soon.

 

4. Beyond RCTs – not happening – 1/10

GiveWell keeps setting out plans to expand the scope of their research (see 2018 plans and 2019 plans) and, in their own words they “failed to achieve this goal” (see 2018 review and 2019 review). When asked they said that “We primarily attribute this to having a limited number of staff who were positioned to conduct this work, and those staff having many competing demands on their time … we are continuing to hire and expect this will enable us to make additional progress in new areas.” I am not super optimistic given their 2020 plan for new research is less ambitious than previously insofar as it focuses solely on public health.

Open Philanthropy are mostly deferring to GiveWell although they express support of GiveWell’s unmaterialised plans to expand their research and they are funding the Center for Global Development’s policy work. The only useful new research in this space seems to be a small amount of work from Founders Pledge, it is unclear the extent to which they plan to do more work in this area.

 

5. Beyond speculation (practical longtermism) – partially happening – 6/10

The best source of research and experimentation in this space is again OpenPhil. They are experimenting with trying to influence policy related to the far future and doing research on topics relevant to long termism. However as already highlighted it is unclear how OpenPhil are comparing different causes, rather than looking out for giving opportunities across a variety of causes and seeing what they can fund and what the impact of that will be.

The Global Priorities Institute (GPI) are looking to improve the quality of thinking in this space. They have so far produced only philosophy papers. It is useful stuff and valuable for building traction in academia, but personally I am pretty sceptical about humans solving philosophy soon and would rather have some answers within the next few decades.

There are a few others doing small amounts of research on specific topics such as Center on Long Term Risk (CLR) and Future of Humanity Institute (FHI).

Overall there seems to be a lot of longtermsim research but the amount that is going into what you could plausibly call cause prioritisation is small and with the possible, but unclear, exception of OpenPhil progress in this space is minimal.

 

Now this is just one way of thinking through the work I would like to see based on my subjective experiences of navigating this community for the past decade, I am sure this could be done differently but overall I give the EA community a whopping 28% for cause prioritisation research. Better than Titanic II (tagline: they said it couldn't happen twice) but not quite as good as The Emoji Movie

 

In short:

There is not nearly enough work in this space.

 

 

D. Why is this underinvested in and next steps

I think that this space needs new organisations (and/or existing organisations to significantly refocus in this direction). But before you swallow everything I have said hook line and sinker and head off to start a cause prioritisation organisation I think we need to examine why this work might be underinvested in and what we can learn.

In the order that I think is important, some of the challenges are:

 

1. It is unclear what the theory of change would be for research organisations in this space.

Different organisations have different theories of change for research.

  • For a big funder (like Open Philanthropy) the theory of change is:
    do research → shift money.
  • For individual academics the theory of change is:
    do research → get published + have imapct.
  • For organisations with a big audience (like 80000 Hours) the theory of change is:
    do research → influence audience.

But for a new organisation to solely focus on doing the research that they believed would be most useful for improving the world it is unclear what the theory of change would be. Some options are:

  • Do research → build audience on quality of research → then influence audience
  • Do research + persuade other organisations to use your research → influence their audiences and money

These paths are valid but they have a difficult extra step. Any organisation entering this space needs to be doing multiple things at once and needs to convince funders that they can create value from the research. For example Let’s Fund has done some useful research but struggled to demonstrate that they can turn research into money moved.

I do not have a magic solution to this. Ideally a new organisation in this space would have enough initial cause neutral funding to allow a reasonable amount of research to be done to demonstrate effectiveness. One idea is to have some level of pre-commitment from a large funder (or from an organisation such as OpenPhil or 80K) that they would use the research. Another idea is to have good influencers on board at the start, for example for policy research having a ex-senior politician on board could help make the case your research would be noticed – the Copenhagen Consensus seemed to start this way.

(Also, I have never worked in academia so there may be theories of change in the academic space that others could identify.)

 

2. It is difficult to compete with the existing organisations that are just not quite doing this.

I think one of the reasons why not enough has been done in this space is that organisations and individuals reach conclusions about what is most important for themselves (not necessarily in a way that is convincing to others) and then choose to focus on that.

For example 80000 Hours have [edited: focused on specific] priority paths. The Future of Humanity Institute has focused heavily on AI, setting up the Centre for the Governance of AI. Even GiveWell used to have a broader remit before they focused in on global health. (There are of course advantages to focus. For example GiveWell’s focus led to them significantly improving their charity recommendations, they no longer recommend terrible approaches like microfinance, but it has limited exploration.)

I think that people are hesitant to do something new if they think it is being done, and funders want to know why the new thing is different so the abundance of organisations that used to do cause prioritisation research or do research that is a subcategory of cause prioritisation research limits other organisations from starting up.

My solution to this is to write this post to convince others that this work is not being done.

 

3. This work is not intractable but it is difficult

This work is difficult. It is not like standard academic research as it needs to pull in a vast variety of different areas and topics, from ethics, to economics, to history, to international relations. Finding polymaths to compare across different interventions of different types is very difficult.

For example finding good staff has clearly impacted GiveWell’s ability to expand their research.

I suggest new organisations in this space might want to consider working differently, for example having a large budget for contracting top quality research across different fields and lower numbers of paid staff.

I also suggest interdisciplinary input into drafting research agendas. (One economics student told me that when reading the GPI research agenda, the economics parts read like it was written by philosophers. Maybe this contributes to the lack of headway on their economics research plans.)

When drafting this post I began to wonder if such research is actually intractable. I think Paul’s arguments counter this somewhat but the thing that gives me the most hope is that some of the best research in this space appears to be random posts from individuals on the EA forum. For example Growth and the case against randomista development, Reducing long-term risks from malevolent actors (part funded by CLR) Does climate change deserve more attention within EA, Increasing Access to Pain Relief in Developing Countries, High Time For Drug Policy Reform. I am also impressed with new organisations such as the fledgling Happier Lives Institute who are challenging the way we think about wellbeing. This makes me think there is likely a lot of tractable important cause prioritisation research that could be done and the problem is a lack of effort not tractability.

 

4. It is difficult to find cause neutral funding.

I think funders like to choose their cause and stick with it so there is a lack of cause neutral funding. 

For example Rethink Priorities looked really exciting when it got started with their co-founder expressing strong support for practical prioritisation research. But their research has mostly focused on animal welfare interventions, not on comparing between causes. They cite having to follow the funding as the main reason for this.

I think funders who have benefited from cause prioritisation research done to date should apportion a chunk of their future funding to support more such research.

 

In short 

There are a bunch of barriers to good cause prioritisation research. But I believe they are all overcomeable, and they do not make a strong case that such research is intractable.

 

 

Conclusion

So there we have it dear reader my musing and thoughts on cause prioritisation, mixed in with a broad undercurrent of dissatisfaction with the EA community. Maybe I am just more jaded in my old age (early 30s) but I think I was more optimistic about the intellectual direction of the EA community when it had no power or influence nearly a decade ago. Intellectual progress in the field of doing good has been much slower than I hoped.

But I am an optimistic fellow. I do think we can make progress. There has been just enough traction to give me hope. It just needs a bit more effort, a bit more searching.

So my request to you. Either disagree with me, tell me that sufficient progress is happening, or change how you act in some small way. Be a bit more uncertain, a bit more willing to donate to fund or to go into cause prioritisation research. And if you work in an EA org please stop focusing so much on the cause areas you each believe are most important and increase the amount of cause neutral work and funding that you do.

I am considering starting a new organisation in this space with a focus on policy interventions. If you want to be involved or have ideas, or have some reason to think this is not actually a good use of my time, then comment below or message me. 

And do comment. I want your thoughts big or small. Most of my recent posts on this forum had minimal comments.

 

Did you read the post by CarlaZoeC that I linked to above? I hope not because they write better than me so I am going to end by stealing their conclusion:

“EA is not your average activist group on the market-place on ideas on how to live. It has announced far greater ambitions: to research humanity’s future, to reduce sentient suffering and to navigate towards a stable world” 

“But if the ambition is great, the intellectual standards must match it. … Humanity lacks clarity on the nature of the Good, what constitutes a mature civilization or how to use technology. In contrast, EA appears to have suspiciously concrete answers.”

“I wish EA would more visibly respect the uncertainty they deal in. Indeed, some EAs are exemplary - some wear uncertainty like a badge of honour.... For them, EA is a quest, an attempt to approach big questions of valuable futures, existential risk and the good life, rather than implementing an answer. I wish this would be the norm. I wish all would enjoy and commit to the search, instead of pledging allegiance to preliminary answers. … [it is like that that we] have the best chance of succeeding in the EA quest.” 

 

 

 

FOOTNOTES

[1] This is based on my experience of diving into a range of activism spaces, charity projects and other assorted communities of people trying to do good. It is very rare for people to think strategically about what to focus on to the most good. GiveWell also make the case that charitable foundations tend not to think this way in this post.

[2] This experience did lead me to start an EA London charity evaluation giving circle for people who had strong moral intuitions that equality and justice were of value. Write up here.

[3] This sentence is a quote from the discussion about the value of diversity in the most recent 80K podcast. But for more on this I also recommend checking out In Defence of Epistemic Modesty.

[4] I accept this is somewhat caricatured, but I maintain that many people in EA fall close to these archetypes. (Except for the effective animal activism folk who nicely bridge this gap, maybe I should just go join them.)

[5] Look out for my upcoming report with CSER on this topic

228

83 comments, sorted by Highlighting new comments since Today at 11:28 AM
New Comment
Some comments are truncated due to high volume. (⌘F to expand all)Change truncation settings

Thanks, I definitely agree that there should be more prioritization research. (I work at GPI, so maybe that’s predictable.) And I agree that for all the EA talk about how important it is, there's surprisingly little really being done.

One point I'd like to raise, though: I don’t know what you’re looking for exactly, but my impression is that good prioritization research will in general not resemble what EA people usually have in mind when they talk about “cause prioritization”. So when putting together an overview like this, one might overlook some of even what little prioritization research is being done.

In my experience, people usually imagine a process of explicitly listing causes, thinking through and evaluating the consequences of working in each of them, and then ranking the results (kind of like GiveWell does with global poverty charities). I expect that the main reason more of this doesn’t exist is that, when people try to start doing this, they typically conclude it isn’t actually the most helpful way to shed light on which cause EA actors should focus on.

I think that, more often than not, a more helpful way to go a... (read more)

Hey Phil. I'm someone who is very interested in the work of GPI and am impressed by what I have seen so far. I'm looking forward to seeing what the new economists get up to!

I had a look at Leopold's paper a while back, have listened to you on the 80K podcast and have watched a few of GPI's videos including Christian Tarsney's one on the epistemic challenge to longtermism. I notice that in a lot of this research, key results are highly sensitive to the value of certain parameters. My memory is slightly hazy on specifics but I think in Christian's paper the validity of longtermism depends largely on the existence and frequency of exogenous nullifying events (ENEs) that can essentially wipe out any trajectory change efforts that came before (apologies if I'm not being perfectly accurate here).

I am wondering if empirical estimation of key parameters is a gap in current cause prioritisation research. Because the value of these parameters is so important in determining results from the models, it seems very high-value to more accurately estimate these parameters. Do you know if anyone is actually doing this? Is anyone for example looking into the nature of ENEs? Is this something new economists at GPI might engage in? If this type of research isn't suitable for GPI, does GPI need closer links to other research institutions that are interested in carrying out more empirical research?

2trammell4moThanks! I agree that people in EA—including Christian, Leopold, and myself—have done a fair bit of theory/modeling work at this point which would benefit from relevant empirical work. I don’t think this is what either of the current new economists will engage in anytime soon, unfortunately. But I don’t think it would be outside a GPI economist’s remit, especially once we’ve grown.
2jackmalde4moOK that’s good to hear. It probably makes sense to spend some time laying a solid theoretical base to build on. I’m aware of how new GPI still is so I’m looking forward to seeing how things progress!
9weeatquince4moHi, Thank you for this really helpful comment. It was really interesting to read about how you work on cause prioritisation research and use IAMs. Glad that GPI will be expanding.
7FCCC3moI think this is one of the most important things we can be doing. Maybe even the most important since it covers such a wide area and so much government policy is so far from optimal. I don't think that's right. I've written about what it means for a system to do "the optimal thing" [https://forum.effectivealtruism.org/posts/FhkXvdP6Dy9BLcJFc/] and the answer cannot be that a single policy maximizes your objective function: Unless by policy, you mean "the entirety of what government does", then yes. But given that you're going to consider one area at a time, and you're "only including all the levers between which you’re considering", you could reach a local optimum rather than a truly ideal end state. The way I like to think about it is "How would a system for prisons (for example) be in the best possible future?" This is not necessarily going to be the system that does the greatest good at the margin when constrained to the domain you're considering (though they often are). Rather than think about a system maximizing your objective function, it's better to think of systems as satisfying goals that are aligned with your objective function.
3evelynciara3moI wonder if we could create an open source library of IAMs for researchers and EAs to use and audit.
5Milan_Griffes3moAt a glance, Salesforce's AI Economist [https://marginalrevolution.com/marginalrevolution/2020/08/a-real-world-ai-economist.html] seems like an attempted implementation of an IAM.

Thanks for the post! Much of it resonated with me.

A few quick thoughts:

1. I could see some reads of this being something like, "EA researchers are doing a bad job and should feel bad." I wouldn't agree with this (mainly the latter bit) and assume the author wouldn't either. Lots of EAs I know seem to be doing about the best that they know of and have a lot of challenges they are working to overcome. 

2. I've had some similar frustrations over the last few years. I think that there is a fair bit of obvious cause prioritization research to be done that's getting relatively little attention. I'm not as confident as you seem to be about this, but agree it seems to be an issue.

3. I would categorize many of the issues as being systematic between different sectors. I think significant effort in these areas would require bold efforts with significant human and financial capital, and these clusters are rare. Right now the funding situation is still quite messy for ventures outside the core OpenPhil cause areas.

I could see an academic initiative taking some of them on, but that would be a significant undertaking from at least one senior academic who may have to take a major risk to do so

... (read more)

Tank you Ozzie. Very very helpful. To respond.

1. EA researchers are doing a great job. Much kudos to them. Fully agree with you on that. I think this is mostly a coordination issue. 

3. Agree a messy funding situation is a problem. Not so sure there is that big huge gap between groups funded by EA Funds and groups funded by OpenPhil.

4. Maybe we should worry less about "groups doing a bad job at these topics could be net negative". I am not a big donor so find this hard to judge this well. Also I am all for funding well evidenced projects (see my skepticism below about funding "smart young people"). But I am not convinced that we should be that worried that research on this will lead to harm, except in a few very specific cases. Poor research will likely just be ignored. Also most Foundations vet staff more carefully than they vet projects they fund.

5-6. Agree research leaders are rare (hopefully this inspires them). Disagree that junior researchers are rare. You said: "We only have so many strong EA researchers, and fewer people capable of leading teams and obtaining funding." + "It seems really difficult to convince committed researchers to change fields" Very good points. Tha

... (read more)
9Ozzie Gooen4moThanks for the response! Quick responses: 4. I haven't investigated this much myself, I was relaying what I know from donors (I don't donate myself). I've heard a few times that OpenPhil and some of the donors behind EA Funds are quite worried about negative effects. My impression is that the reason for some of this is simple, but there are some more complicated reasons that go into the thinking here that haven't been written up fully. I think Oliver Habryka has a bunch of views here. 5-6. I didn't mean to imply that junior researchers are "rare", just that they are limited in number (which is obvious). My impression is that there's currently a bottleneck to give the very junior researchers experience and reputability, which is unfortunate. This is evidenced by Rethink's round. I think there may be a fair amount of variation in these researchers though; that only a few are really the kinds who could pioneer a new area (this requires a lot of skills and special career risks). 7. I'm also really unsure about this. Though to be fair, I'm unsure about a lot of things. To be clear though, I think that there are probably rather few people this would be a good fit for. I'm really curious just how impressive the original EA founders were compared to all the new EAs. There are way more young EAs now than there were in the early days, so theoretically we should expect that some will be in many ways more competent than the original EA founders, minus in experience of course. Part of me wonders: if we don't see a few obvious candidates for young EA researchers as influential as the founders were, in the next few years, maybe something is going quite wrong. My guess is that we should aim to resemble other groups that are very meritocratic in terms of general leadership and research. 8. Happy to discuss in person. They would take a while to organize and write up. The very simple thing here is that to me, we really could use "funding work" of all types. OpenPhil still emp
8MichaelDickens4moExcellent comment. Do you have a list of the top research areas you'd like to see that aren't getting done? I agree. Forecasting is a common good to many causes, so you'd expect it not to be neglected. But in practice, it seems the only people working on forecasting are EA or EA-adjacent (I'd count Tetlock as adjacent). Recently I've had many empirical questions about the future that I thought could use good forecasts, e.g., for this essay I wrote [https://forum.effectivealtruism.org/posts/3QhcSxHTz2F7xxXdY/estimating-the-philanthropic-discount-rate] , I made some Metaculus questions and used those to help inform the essay. It would be really helpful if it were easier to get good forecasts.

Do you have a list of the top research areas you'd like to see that aren't getting done?

Oh boy. I've had a bunch of things in the back of my mind. Some of this is kind of personal (specific to my own high level beliefs, but wouldn't apply to many others).
I'm a longtermist and believe that most of the expected value will happen in the far future. Because of that, many of the existing global poverty, animal welfare, and criminal justice reform interventions don't seem particularly exciting to me. I'm unsure what to think of AI Risk, but "unsure" is much, much better than "seems highly unlikely." I think it's safe to have some great people here; but currently get the impression that a huge number of EAs are getting into this field, and this seems like too many to me on the margin.

What I'm getting to is: when you exclude most of poverty, animal welfare, criminal justice reform, and AI, there's not a huge amount getting worked on in EA at the moment.

I think I don't quite buy the argument that the only long-term interventions to consider are ones that will cause X-risks in the next ~30 years, nor the argument that the only interventions are ones that will cause X-risks. I think it's fair

... (read more)

This is a really good comment.

  • A lot of the prioritization work, even of, "Let's just estimate a lot of things to get expected values."

I would like to see more of this, and I would also like to see people be less uniformly critical of this sort of work. I've written a few things like this, and I inevitably get a few comments along the lines of, "This estimate isn't actually accurate, you can't know the true expected value, this research is a waste of time." IME I get much more strongly negative comments when I write anything quantitative than when I don't. But I might just be noticing that type of criticism more than other types.

  • Much better EA funding infrastructure, in part for long-term funding.

The rate of institutional value drift is something like 0.5%. Halving this would be extremely beneficial for anyone who wants to invest their money for future generations. It seems likely that if we put more effort into designing stable institutions, we could create EA investment funds that last for much longer.

The rate of individual value drift is even higher, something around 5%. That's really bad. Is there anything we can do about it? Is bringing new people into the movement

... (read more)
8Ozzie Gooen3moI haven't seen these specific examples, but there definitely seems to be a similar bias in other groups. Many organizations are afraid to make any kinds of estimates at all. At the extreme end are people who don't even make clear statements, they just speak in vague metaphors or business jargon that are easy to defend but don't actually convey any information. Needless to say, I think this is an anti-pattern. I'd be curious if anyone reading this would argue. It seems to me like some modeling here would be highly useful, though it can get kind of awkward. I imagine many decent attempts would include numbers like, "total expected benefit of one member". Our culture often finds some of these calculations too "cold and calculating." It could be worth it for someone to do a decent job at some of this, and just publicly write up the main takeaways. I find the ideas you presented quite interesting and reasonable, I'd love to see more work along those lines.
8MichaelA3moI think it would depend a lot on how we operationalise the stance you're arguing in favour of. Overall, at the margin, I'm in favour of: * less use of vague-yet-defensible language * EAs/people in general making and using more explicit, quantitative estimates (including probability estimates) (I'm in favour of these things both in general and when it comes to cause priorisation work.) But I'm somewhat tentative/moderate in those views. For the sake of conversation, I'll skip stating the arguments in favour of those views, and just focus on the arguments against (or the arguments for tentativenesss/moderation). Essentially, as I outlined in this post [https://forum.effectivealtruism.org/posts/KfqFLDkoccf8NQsQe/potential-downsides-of-using-explicit-probabilities] (which I know you already read and left useful comments on), I think making, using, and making public quantitative estimates might sometimes: 1. Cost more time and effort than alternative approaches (such as more qualitative, "all-things-considered" assessments/discussions) 2. Exclude some of the estimators' knowledge (which could’ve been leveraged by alternative approaches) 3. Cause overconfidence and/or cause underestimations of the value of information 4. Succumb to the optimizer’s curse 5. Cause anchoring 6. Cause reputational issues (These downsides won't always occur, can sometimes occur more strongly if we use approaches other than quantitative estimates, and can be outweighed by the benefits of quantitative estimates. But here I'm just focusing on "arguments against".) As a result: * I don't think we should always aim for or require quantitative estimates (including in cause prioritisation work) * I think it may often be wise to combine use of quantitative estimates, formal models, etc. with more intuitive / all-things-considered / "black-box" approaches (see also [https://blog.givewell.org/2014/06/10/sequence-thinking-vs-cluster-thinking/] ) * I defi
6Ozzie Gooen3moNice post. I think I agree with all of that. I'm not advocating for "poorly done quantitative estimates." I think anyone reasonable would admit that it's possible to bungle them. I'm definitely not happy with a local optimum of "not having estimates". It's possible that "having a few estimates" can be worse, but I imagine we'll want to get to the point of "having lots of estimates, and becoming more mature to be able to handle them." at some point, so that's the direction to aim for.
4MichaelA3moI think the "local vs global optima" framing is an interesting way of looking at it. That reminds me of some of my thinking when I was trying to work out whether it'd be net positive to make that database of existential risk estimates (vs it being net negative due to anchoring, reputational issues to EA/longtermists, etc.). In particular, a big part of my reasoning was something like: With your comment in mind, I'd now add:
4MichaelA3moI think this is a good point. A three-factor model of community building [https://www.centreforeffectivealtruism.org/a-three-factor-model-of-community-building/] comes to mind as a prior post that had to tackle and communicate about this sort of tricky thing, and that did a good job of that, in my opinion. That post might be useful reading for other people who have to tackle and communicate about this sort of tricky issue in future. (E.g., I quoted it in a recent post of mine [https://forum.effectivealtruism.org/posts/4oGYbvcy2SRHTWgWk/improving-the-future-by-influencing-actors-benevolence] .) The most relevant parts of that post are the section on "Elitism vs. egalitarianism", and the following paragraph:
2Ozzie Gooen3moThanks!
4MichaelDickens3moThe basic model is really easy. Total number of community members at time t is e (r−v)t, where r is the movement growth rate and v is the value drift rate. So if the value of the EA community is proportional to the number of members, then increasing r by some number of percentage points is exactly as good as decreasing v by the same amount. It's less obvious how to model the tractability of changing r and v.
4Milan_Griffes3moReminds me of the thing where corporations don't want to implement internal prediction markets because implementing a market isn't in the self-interest of any individual decision-maker.
2Ozzie Gooen3moYea, I think there are similar incentives at play in both cases
4MichaelA3moI liked this comment. Do you mean "If you accept that improving the long-term value of the future is more important than reducing extinction risk" (as distinct from existential risk more broadly, which already includes [(rather than existential risk more broadly, which already includes other ways of improving the value of the future)] other ways of improving the value of the future)? Or "If you accept that improving the long-term value of the future is more important than reducing the risk of existential catastrophe in the relatively near future?" Or something else (e.g., about smaller trajectory changes [https://www.lesswrong.com/posts/5czcpvqZ4RH7orcAa/a-proposed-adjustment-to-the-astronomical-waste-argument] )?
4MichaelDickens3moI meant to distinguish between long-term efforts and reducing x-risk in the relatively near future (the second case on your list), sorry that was unclear.

Here's a list I came up with from thinking about this for ~30 minutes:

Better ways of measuring what matters


Help EAs see more clearly, unpack + resolve personal traumas, and boost their efficacy + motivation

  • Emotional healing as a prerequisite to rationality
  • CFAR, OAK, Leverage, etc.
  • Plus building methods to audit which projects are working, which are failing, which are stagnating
  • Perhaps also a data collection project that vacuums up outcomes from the object-level projects?

Strengthen EA community ties / our sense of fellowship

  • More honesty about how weird effective research methods can be
  • More acknowledgement of the interdependent causal complex that gives rise to good research (e.g. Alex Flint's introduction here)
  • More Ben Franklin-esque Juntos
  • Import more of Silicon Valley's "pay it forward" culture
  • Less reputation management / more psychological safety
  • Less sniping
  • OAK, Bay Area group houses, EA Hotel
  • Again, building out (non-dominating) ways to audit & collect data from the object-level projects

Less scrupulosity

  • Tie
... (read more)

That's an interesting list, especially for 30 minutes :) (Makes me wonder what you or others could do with more time.)

Much of it focused on EA community stuff. I kind of wonder if funders are extra resistant to some of this because it seems like they're just "giving money to their friends", which in some ways, they are. I could see some of it feeling odd and looking bad, but I think if done well it could be highly effective.

Many religious and ethnic groups spend a lot of attention helping each other, and it seems to have very positive effects. Right now EA (and the subcommunities I know of in EA) seem fairly far from that still.

https://www.nationalgeographic.com/culture/2018/09/south-asia-america-motels-immigration/

A semi-related point on that topic; I've noticed that for many intelligent EAs, it feels like EA is a competition, not a collaboration. Individuals at social events will be trying to one-up each other with their cleverness. I'm sure I've contributed to this. I've noticed myself becoming jealous when I hear of others who are similar in some ways doing well, which really should make no sense at all. I think in the anonymous surveys 80K did a while back a bunch of people complained that there was a lot of signaling going on and that status was a big deal.

Many companies and open source projects live or die depending on the cultural health. Investments in the cultural health of EA may be difficult to measure, but pay off heavily in the long run.

8Milan_Griffes3moThanks! 100% agree that cultural health is very important, and that EA is under-investing in it. (The "we don't want to just give money to our friends" point resonates, and other scrupulosity-related stuff is probably at play here as well.) Thank you for talking about this! I've noticed similar patterns in my own mind, especially around how I engage with this Forum. (I've been stepping back from it more this year because I've noticed that a lot of my engagement wasn't coming from a loving place.) These dynamics may not make any sense, but there are deep biological & psychological forces giving rise to them. [insert Robin Hanson's "everything you do is signaling" rant here] Right. Last year concerns about status made a lot of heat on the Forum (1 [https://forum.effectivealtruism.org/posts/jmbP9rwXncfa32seH/after-one-year-of-applying-for-ea-jobs-it-is-really-really] , 2 [https://forum.effectivealtruism.org/posts/vMpuXz2zqS8iHya7i/ea-jobs-provide-scarce-non-monetary-goods] , 3 [https://forum.effectivealtruism.org/posts/Lms9WjQawfqERwjBS/the-career-and-the-community] ), but as far as I know nothing has really changed since then, perhaps other than more folks acknowledging that status is a thing. (Status seems closely related to scrupulosity & to EA being vetting-constrained [https://forum.effectivealtruism.org/posts/G2Pfpkcwv3bJNF8o9/ea-is-vetting-constrained] ; I haven't unpacked this yet.)
5MichaelA3mo(A bunch of those ideas seem interesting, but I'll just comment on the one where I have something to say) This does seem to me like it makes it easy to walk back efforts to make EA sexier, but it doesn't seem like it makes it easy to do it again later in a different way (without the odds of success being impaired by the first attempt). Essentially: * I think we could make EA relatively small/non-prominent/whatever again if we wanted to * But it also seems plausible to me that EA can only make "one big first impression", and that that'll colour a lot of people's perceptions of EA if it tries to make a splash again later (even perhaps 10-30 years later). Put another way: * They might stop thinking about EA if we stop actively reminding them * But then if we start competing for their attention again later they'll be like "Wait, aren't those the people who [whatever impression they got of us the first time]?" Posts that informed my thinking here: * Hard-to-reverse decisions destroy option value [https://forum.effectivealtruism.org/posts/eyDDjYrG3i3PRGxtc/hard-to-reverse-decisions-destroy-option-value] (which I see you also referenced yourself) * The fidelity model of spreading ideas [https://www.centreforeffectivealtruism.org/blog/the-fidelity-model-of-spreading-ideas/] * How valuable is movement growth? [https://www.effectivealtruism.org/articles/how-valuable-is-movement-growth-owen-cotton-barratt/] * Why not to rush to translate effective altruism into other languages [https://forum.effectivealtruism.org/posts/Z95TxtkjHGPq4TAqY/why-not-to-rush-to-translate-effective-altruism-into-other]
5Milan_Griffes3moYour list reminds me of this thread: What EA Forum posts do you want someone to write? [https://forum.effectivealtruism.org/posts/9AEEqDR8zcQv2XmsB/what-posts-do-you-want-someone-to-write]
8Ozzie Gooen4moI think I've become a bit convinced that incentive and coordination problems are so poor that many "common goods" are surprisingly neglected. The history of the slow development and proliferation of Bayesian techniques in general (up to around 20 years ago maybe, but even now I think the foundations can be improved a lot) seems quite awful. Also, at this point, I feel quite strong about much of the EA community; like we've gathered up many of the most [intelligent + pragmatic + agentic + high-level-optimizing] people in the world. As such I think we can compete and do a good job in many areas we may choose to focus on. So it could be that we could move up from "absolutely, incredibly neglected", to "just somewhat neglected", which could open up a whole bunch of fields.
9MichaelDickens4moIt seems like I routinely learn about some smart and insightful person through non-EA channels and then later find out they're involved in EA or at least subscribe to EA principles—most recent example for me is Gordon Irlam [https://www.gordoni.com/], who I originally learned about through his writings on portfolio selection.
8Ozzie Gooen4moI've been thinking a lot about the lack of non-EA interest or focus on forecasting or related tools. I was very surprised when I made Guesstimate and there was both excitement from several people, but not that much excitement from most businesses or governments. I think that forecasting of the GJP sort is still highly niche. Almost no one knows of it or understands the value. You can look at this as similar to specific advances in, say, type theory or information theory. The really smart groups that have interests in improving their long term judgement seem to be financial institutions and similar. These are both highly secretive, and not interested in spending extra effort helping outside groups. So to really advance a field like judgemental forecasting would require a combination of expertise, funding, and interest in helping the broad public, and this is a highly unusual combination. I imagine that if IARPA wasn't around in time to both be interested in and able to fund GJP's efforts, much less would have happened there. I'd also personally point out that I'd expect that IARPA's funding of it was around 1/3rd or maybe 1/20th as efficient as it would have been if OpenPhil would have organized a more directed effort, in terms of global benefit. This makes me think that there are probably many other very specific technology and research efforts that also be exciting for us to focus on, but we don't have the expertise to recognize them. May may have gotten lucky with forecasting/estimation tech, as that was something we had to get close to anyway for other reasons.
6MichaelDickens3moAlso worth noting that the managing director of IARPA's forecasting program was Jason Matheny, who previously founded New Harvest (which does cultured meat research, and was the first such org AFAIK) and did x-risk research at FHI.
4Ozzie Gooen3moYep, and a few others at IARPA who worked around the forecasting stuff were also EAs or close.

Thanks for this, it's pretty interesting to get your perspective as someone who's been (I presume) heavily engaged in the community for some time. I thought your other post on the All-Party Parliamentary Group for Future Generations was awesome, by the way.

You asked for comments including "small" thoughts so here are some from me, for what they're worth. These are my current views which I can easily see changing if I were to think about this more etc.

I think I basically agree that there doesn't seem to have been much progress in cause prioritisation in say the last five years, compared to what you might have hoped for.

(mainly written to clarify my own thoughts:) It seems like you can do cause prioritisation work either by comparing different causes, or by investigating a particular cause (especially a cause that's relatively unknown or poorly investigated), or by doing more "foundational" things like asking "what is moral value anyway?", "how should we compare options under uncertainty", etc.

My impression the Effective Altruism community has invested a significant amount of resources into cause prioritisation research, and relative lack of progress is because it's hard

  • The Global
... (read more)

I agree that the cause prioritisation work we need to do now is far harder than the work we were doing ten years ago. I think AI Impacts provides an interesting illustration of that: It was initially set up essentially as a cause prioritisation org. But in doing that work it became clear that whereas in comparing between different global development interventions there was a large published literature to build on, when trying to compare work on AI to other areas, and compare interventions within AI safety, there was far less to go on. That led to the conclusion that the work they should do first was get a better grasp on questions like 'how fast will AI likely develop, and how discontinuously?'.

I think another thing going on is that the stakes have become higher. When Giving What We Can first started publishing recommendations eg comparing between donating to education or deworming, we only had ~30 members. That's a lot of money over people's lifetimes, but it's nowhere near the resources the EA movement now commands. The huge increase in resources to allocate makes it more worth doing the foundational work that groups like AI Impacts do, and also the theoretic work GPI does. I think that makes it look like there's less work being done, because there are way fewer actionable results per hour spent.

6weeatquince4moHi Ben. Thank you for this. This is exactly what I like, people replying with their impressions of the post, even if rough, so that I get some idea of how people feel and if this resonates. So thank you. - - That said I disagree with your claim. You say "I think it's just very hard and that this explains a lot of what you're describing". I think it may well be difficult but it is mostly not happening due to underinvestment and lack of coordination in this space. Hence raising a flag. I make this case above by comparing what I would see as a good coverage of the space with what is actually happening, so don’t have much to add here except that it is interesting that others see it differently. I note a few counterexamples to the idea it is not done because it is hard (even in the "longtermist" area) such as: 80K's stated reason [https://forum.effectivealtruism.org/posts/dC8w35Y9G7gyWhNHK/what-will-80-000-hours-provide-and-not-provide-within-the] for doing less in this space is that they have reached a conclusion (priority paths) that they are happy with, that GPI was only created recently (research agenda is from 2019 [https://forum.effectivealtruism.org/posts/cpJRB7thJpESTquBK/introducing-gpi-s-new-research-agenda] ), Rethink Priorities is following funding, AI strategy is also difficult but is progressing much quicker. etc. - - Overall, I don’t have a strong view on this, and maybe you are correct. But this is something that could be looked into more. In particular I have mostly dug into research on websites but if I (or anyone) had more time it would be great talk to people who have worked on this and see if it is difficult or underinvested in (or both). I also think you could with a bit of time somewhat address this question by writing a research agenda and looking for potential low hanging research fruit in this domain.

Hey Sam, just a very quick comment that the post you link to wasn't meant to imply we intend to do less prioritisation research than before.

The 50/30/20 split we mention there was for how we intend to split delivery efforts across different target audiences, rather than on research vs. delivery. And also note that this means ~50% of effort is going into non-priority paths, which will include new potential priorities & career paths (such as the lists we posted recently).

As Rob notes in another comment, we still intend to spend ~10% of team time on research, similar to the past, and more total time because the team is larger. This would include looking into whether we should add new priority paths or problem areas.

6weeatquince4moHi Ben, Thank you for flagging – it is super amazing to hear and very excited by that. I looked at a lot of organisations and tried to extrapolate what they will be doing in this space from the public information rather than reaching out, so it is great to see comments saying that research along these lines will be happening, and sorry for any thing mischaracterised.
2weeatquince4moThis comment below is also relevant: https://forum.effectivealtruism.org/posts/MSYhEatxkEfg46j3D/the-case-of-the-missing-cause-prioritisation-research?commentId=RGX9f6PXvWkBvCEoK [https://forum.effectivealtruism.org/posts/MSYhEatxkEfg46j3D/the-case-of-the-missing-cause-prioritisation-research?commentId=RGX9f6PXvWkBvCEoK]

Thank you for writing this!
I think your analysis can be specifically useful for people who want to contribute and feel like they're not sure where to look for neglected areas in EA.

I'll add a small comment regarding "It is difficult to compete with the existing organisations that are just not quite doing this":

My experience with orgs in the EA community is that pretty much everyone is incredibly cooperative and genuinely happy to see others fill in the gaps that they're leaving.
I've been in talks with 80,000 hours and a few other orgs about an initiative in the careers space for a while now. Everyone we've talked to was both open about what they're doing (and what they aren't doing) and ridiculously helpful with advice and support.

I think if someone is serious about trying to fill a gap in the EA body of work: It's important to understand from adjacent orgs how big \ real this gap is and if they have comments about your approach to it. And while I can see why someone would be worried, I think if you approach with the right attitude, the 'competition' would have far more benefits than harms.

Thank you for this comment. I fully agree with this and would say that my experience of the EA community is a very positive one and that the EA community and EA organisations work very well together and are very willing to share ideas, talk and support one another. I am sure would be much support for anyone trying to fill these gaps.

Thanks for writing the post! I think we need a lot more strategy research, cause prioritization being one of the most important types, and that is why we founded Convergence Analysis (theory of change and strategy, our site, and our publications). Within our focus of x-risk reduction we do cause prioritization, describe how to do strategy research, and have been working to fill the EA information hazard policy gap. We are mostly focused on strategy research as a whole which lays the groundwork for cause prioritization. Here are some of our articles:

We’re small and relatively new group and we’d like to see more people and groups do this type of research and that this field get more support and grow. There is a vast amount to do and immense opportunity in doing good with this type of research.

9Ozzie Gooen4moI'll give a +1 for Convergence. I've known the team for a while and worked with Justin a few years back. It's a bit on the theoretical side of prioritization, but that sort of thinking often does lead to more immediate value. My impression is also that more funding could be quite useful to them, if anyone is reading this considering.

"For example 80000 Hours have stopped cause prioritisation work to focus on their priority paths"

Hey Sam — being a small organisation 80,000 Hours has only ever had fairly limited staff time for cause priorities research.

But I wouldn't say we're doing less of it than before, and we haven't decided to cut it. For instance see Arden Koehler's recent posts about Ideas for high impact careers beyond our priority paths and Global issues beyond 80,000 Hours’ current priorities.

We aim to put ~10% of team time into underlying research, where one topic is trying to figure out which problems and paths go into each priority level. We also have podcast episodes on newer problems from time to time.

All that said, I am sympathetic to the idea that as a community we are underinvesting in cause priorities research.

9weeatquince4moSuper great to hear that 10% of 80000 Hours team time will go into underlying research. (Also apologies for getting things wrong, was generalising from what I could find online about what 80K plans to work on – have edited the post). If you have more info on what this research might look into do let me know. – – That there is an exploit explore tradeoff. Continuing to do cause prioritisation research needs to be weighed against focusing on specific cause areas. I imply in my post that EA organisations have jumped too quickly into exploit. (I mention 80K and FHI, but l am judging from an outside view so might be wrong). I think this is a hard case to make, especially to anyone who is more certain than me about which causes matter (which may be the most EA folk). That said there are other reasons for continuing to explore, to create a diverse community, epistemic humility, game theoretic reasons (better if everyone explores a bit more), to counter optimism bias, etc. Not sure I am explaining this well. I guess I am saying that I still think the high level point I was making stands: that EA organisations seem to move towards exploit quicker than I would like. But do let me know if you disagree.

Thanks for making this post, I think this sort of discussion is very important.

It seems to me (predictably given the introduction) that far and away the most valuable thing EA has done is the development of and promotion of cause prioritisation as a concept.

I disagree with this. Here's an alternative framing:

  • EA's big ethical ideas are 1) reviving strong, active, personal moral duties, 2) longtermism, 3) some practical implications of welfarism that academic philosophy has largely overlooked (e.g. the moral importance of wild animal suffering, mental health, simulated consciousnesses, etc).
  • I don't think EA has had many big empirical ideas (by which I mean ideas about how the world works, not just ideas involving experimentation and observation). We've adopted some views about AI from rationalists (imo without building on them much so far, although that's changing), some views about futurism from transhumanists, and some views about global development from economists. Of course there's a lot of people in those groups who are also EAs, but it doesn't feel like many of these ideas have been developed "under the banner of EA".

When I think ab... (read more)

1tamgent3moAiming for maxipok doesn't mean not influencing the trajectory (if the counterfactual is catastrophe), it's just much harder to measure impact. If measuring impact is hard, de-risking becomes more important, because of path-dependency. If we build out one or two particular longtermist cause areas really strongly with lots of certainty, they'll have a lot of momentum (orgs and stuff) and if we find out later that they are having negative impact or not having impact (or worse, this happens and we just never find out), that will be bad. I agree longtermist cause prioritisation is harder, even though I didn't really think your reasons were very well articulated (in particular I don't understand why you're comparing altruism with understanding & controlling the future, seems like apples and oranges to me and surely it's the intersection of X and altruism with the market gap), but I don't think it's less valuable.

I don't share your optimistic view of research. You write:

it is reasonable to think that research would make progress because:
Very little research has been done on this so far.

That's because cause prioritization research is extremely difficult, not because no one has thought to do this.

Human history reflects positively on our ability to build a collective understanding of a difficult subject and eventually make headway.

Survivorship bias: what about all of the difficult subjects where we couldn't make any progress and gave up?

Even if difficult, we should at least try! We would learn why such research is hard and should keep going until we reach a point of diminishing returns.

No, we should try if the expected returns are better than the next alternative. What if we've already hit diminishing returns?

More generally, research isn't magic. Hiring a researcher and having them work 9-5 is no guarantee of solving a problem. You write:

What empirical evidence is there that we can reliably impact the long run trajectory of humanity and how have similar efforts gone in the past? [...]
I think there needs to be much better research into how to make complex decisions despite high uncertainty.

Isn't it obvious that allocating researcher hours to these questions would be a waste of money? Almost by definition, we can't have good evidence that we can impact the long-run (ie. centuries) trajectory of humanity, because we haven't been collecting data for that long. And making complex decisions under high uncertainty will always be incredibly difficult; in the best case scenario, more research might yield small improvements in decision-making.

Hi Michael. Thank you for your points. It is good to hear opposing views. I have never worked in pure research so find it hard to judge and somewhat parroted Paul's post. You may well be correct about the difficulty of research.

Let me try to draw from my own experience to elucidate why I may jumping to different intuitive conclusions on this question

My experience of research is from policy development. I think 2/3 of policy development is super easy and 1/3 is super difficult. The super easy stuff is just looking at the world and seeing if there are answers already out there and implementing them. For example on US police reform or UK tax policy or technology regulatory policy. We mostly know how to do these things well, we just need some incentive to implement best practice. The super difficult stuff is the foundational work, where a new problem emerges and no existing solutions abound, eg financial stability policy.

Now when I look at a question such as the one you quote of "much better research into how to make complex decisions despite high uncertainty" it seems to me to be a mix, but with definite areas that fall more towards the easy side. There appear to be a number of fields

... (read more)
2Michael_Wiebe4moThanks for the reply. I'm a jaded PhD student, but I am open to updating towards research-optimism. I would distinguish research from implementation of research. I agree that there seems to be l0w-hanging fruit in implementing best practices, but I think implementation can be a super difficult problem in its own right. (See the state capacity literature [https://bsc.cid.harvard.edu/].)

Thanks very much for writing this up Sam. Two points from my perspective at the Happier Lives Institute, who you kindly mention and is a new entrant to cause prioritisation work.

First, you say this on theories of change:

But for a new organisation to solely focus on doing the research that they believed would be most useful for improving the world it is unclear what the theory of change would be. Some options are:
Do research → build audience on quality of research → then influence audience
Do research + persuade other organisations to use your research → influence their audiences and money

I think this nails the difficulty for new cause prioritisation research (where 'new' means 'not being done by an existing EA organisation'). The existing organisations are the 'gatekeepers' for resources but doing novel cause prioritsation work requires, of necessity, doing work those organisations themselves consider low-priority (otherwise they would do it themselves). This creates a tension: funders often want potential entrants to show they have 'buy-in' from existing orgs. But the more novel the project, the less 'buy-in' i... (read more)

I agree that setting up new orgs is really challenging. I think this maybe oversells the difficulty of getting buy in from existing orgs in a way that might unduly put people off trying to set up new projects though.

My main experience with this is setting up the Global Priorities Institute. GPI does fairly different work from other EA orgs (though some overlap with FHI), and is much more foundational/theoretic than typical ones. You might expect that to get extra push back from EAs, given that the theory of change is of necessity less direct than for orgs like openphil. I was in the fortunate position of already working with CEA, which ofc made things easier. And getting funding from OpenPhil was definitely a long process. But I actually found it really helpful. The kind of docs etc they asked for were ones that it was useful for us to produce (for example pinning down our vision going forward, including milestones that would indicate we were or weren't on track), and their comments on our strategy and work was helpful for improving them.

I think some things that helped, and that others might find useful, were:

  • Doing a bunch of consultation early on in the process. That impr
... (read more)

For example Rethink Priorities looked really exciting when it got started with their co-founder expressing strong support for practical prioritisation research. But their research has mostly focused on animal welfare interventions, not on comparing between causes.

For what it's worth, Rethink Priorities' research on sentience and capacity for welfare can be used to inform us how to prioritize between interventions for nonhuman animals and interventions for humans. Charity Entrepreneurship has also done research comparing animal welfare under different conditions for different species, including humans, and Founders Pledge has done a sensitivity analysis comparing the Humane League and AMF.

This is a great post - thanks a lot for writing it. I work at GPI, so want to add a bit of context on a couple of points, and add some of my own thoughts. Standard disclaimer that these are my personal views and not those of GPI though. 


First, on GPI's research agenda, and our progress in econ:

"(One economics student told me that when reading the GPI research agenda, the economics parts read like it was written by philosophers. Maybe this contributes to the lack of headway on their economics research plans.)"

I think this is accurate and a reflection of how the research agenda was written and has evolved. For what it's worth, we're currently working on refreshing the research agenda to reflect some of the 'exploration research' we've done in economics in the past ~18 months - we should have an updated version in the next few months. More generally, we've had very little econ research capacity to date beyond pre-doctoral researchers (very junior in academic terms). This will improve very shortly -- as Phil notes in a previous comment, we've hired two postdocs to start in the next month -- but as others have noted, high ... (read more)

2. Different views – not happening – 0/10

For what it's worth, Christian Tarsney from GPI has looked at other aggregative views:

  • Average Utilitarianism Implies Solipsistic Egoism. Summary: average utilitarianism and rank-discounted utilitarianism reduce to egoism due to the possibility of solipsism. Might also apply to variable value theories, depending on the factors. See also the earlier The average utilitarian’s solipsism wager by Caspar Oesterheld.
  • Non-additive axiologies in large worlds. Summary: With large background (e.g. unaffected) populations, average utilitarianism, and some kinds of egalitarian and prioritarian theories reduce to additive theories, i.e. basically utilitarianism. Geometric rank-discounted utilitarianism reduces to maximin instead. (That being said, this doesn't imply we should maximize expected total utility, since it doesn't rule out risk-aversion.)

So, if your population axiology is representable by a single (continuous and impartial) real-valued function of utilities for finite populations (so excluding some person-affecting views), it seems hard to avoid totalism.

Also, I think such views (or utilitarianism) but with deontological constraints are covered

... (read more)
8MichaelStJules3moSome other works and authors exploring other views and their relationship to EA or EA concepts: * Teruji Thomas, 'The Asymmetry, Uncertainty, and the Long Term' [https://forum.effectivealtruism.org/posts/fqynQ4bxsXsAhR79c/teruji-thomas-the-asymmetry-uncertainty-and-the-long-term] (EA Forum post [https://forum.effectivealtruism.org/posts/fqynQ4bxsXsAhR79c/teruji-thomas-the-asymmetry-uncertainty-and-the-long-term] ) * Phil Torres [https://www.xriskology.com/] (overview of focus [https://www.xriskology.com/key-ideas], publications [https://www.xriskology.com/publications], popular media writing [https://www.xriskology.com/mfhf], EA Forum account [https://forum.effectivealtruism.org/users/philosophytorres]), who works on x-risks, but I think believe in virtue ethics, and is critical of total utilitarianism, longtermism and EA's neglect of social justice. * Roger Crisp and Theron Pummer, 'Effective Justice' [https://philarchive.org/rec/CRIEJ-2], discussing "Effective Justice, a possible social movement that would encourage promoting justice most effectively, given limited resources" * Open Phil works on causes that don't receive that much attention within the rest of EA [https://www.openphilanthropy.org/focus]. * Johann Frick, 'On the Survival of Humanity' [https://www.tandfonline.com/doi/abs/10.1080/00455091.2017.1301764] (pdf [https://scholar.princeton.edu/sites/default/files/jfrick/files/survival_of_humanity_feb_17_formatted_final.pdf] ), discussing the "final value of humanity", separate from the (aggregate) value of individuals. * Hilary Greaves, William MacAskill, 'The case for strong longtermism' [https://globalprioritiesinstitute.org/hilary-greaves-william-macaskill-the-case-for-strong-longtermism/] (discusses risk-aversion in 4.2) * GPI's other research [https://globalprioritiesinstitute.org/papers/] on decision theory and cluelessness (deep uncertainty, Knightian uncertainty), offering
5MichaelA3moOn how risk- and uncertainty-aversion should arguably affect EA decisions, this was also this talk [https://globalprioritiesinstitute.org/lara-buchak-should-effective-altruism-focus-on-global-health-or-existential-threats/] hosted by GPI, by Lara Buchak. (I'm mentioning that because it seems relevant, not necessarily because I agreed with the talk or with the basic idea that we should take intrinsic risk- or uncertainty-aversion seriously.)
1Michael_Wiebe3moTangent: What do you mean by this? Isn't risk aversion just a fact about the utility function? You can maximize expected utility no matter how the utility function is shaped.
3MichaelStJules3moAh, we use utility in two ways, the social welfare function whose expected value you maximize, and the welfares of individuals on which your social welfare function depends. You can be a risk-averse utilitarian, for example, with a social welfare function likef(∑iui), where theuiare the individual utilities/welfares andf:R→Ris nondecreasing and concave.
1Michael_Wiebe3moHm, I've never seen the use of $f$ like that. Can you point to an example?
3MichaelStJules3moAn example functionf, or an example where someone actually recommended or used a particular functionf? I don't know of any of the latter, but using an increasing and boundedfhas come up in some discussions about infinite ethics (although it couldn't be concave towards−∞). I discuss bounded utility functions here [https://forum.effectivealtruism.org/posts/ajZ8AxhEtny7Hhbv7/if-you-value-future-people-why-do-you-consider-near-term?commentId=JhWPx3GbXeDRxSd3Z] . An example function is1−e−x. See this link [https://www.wolframalpha.com/input/?i=1-e%5E%28-x%29] for a graph. It's strictly increasing and strictly concave everywhere, and bounded above, but not below.
3Michael_Wiebe3moYes, I meant an example of someone using f in this way. It doesn't seem to be standard in welfare economics [https://en.wikipedia.org/wiki/Social_welfare_function].

I agree wholeheartedly with this! Strong upvote from me.

I agree that cause prioritization research in EA focuses almost entirely on utilitarian and longtermist views. There's substantial diversity of ethical theories within this space, but I bet that most of the world's population are not longtermist utilitarians. I'd like to see more research trying to apply cause prioritization to non-utilitarian worldviews such as ones that emphasize distributive justice.

One thing I notice is that, with few exceptions, the path to change for EA folk who want to improve the long-run future is research. They work at research institutions, design AI systems, fund research, support research. Those that do not do research seem to be trying to accumulate power or wealth or CV points in the vague hope that at some point the researchers will know what needs doing.

Fully agree, but I think it's ironic (in a good way) that your proposed solution is "more global priorities research." When I see some of 80K's more recent advice, I think, "Dude, I already sank 4 years of college into studying CS and training to be a software engineer and now you expect me to shift into research or public policy jobs?" Now

... (read more)
9weeatquince4moHi evelynciara, Thank you so much for your positivity and for complementing my writing. Also to say do not feel discouraged. It is super unclear exactly what the community needs and I we should each be doing what we can with the skills we have and see what form that takes.

Thanks for writing this up! I think you're raising many interesting points, especially about a greater focus on policy and going "beyond speculation".

However, I'm more optimistic than you are about the degree of work invested in cause prioritisation, and the ensuing progress we've seen over the last years. See this recent comment of mine - I'd be curious if you find those examples convincing.

Also, speaking as someone who is working on this myself, there is quite a bit of research on s-risks and cause prioritisation from a suffering-focused perspective, which is one form of "different views" - though perhaps this is not what you had in mind. (I think it might be good to clarify in more detail what sort of work you want to see, because the term "cause prioritisation research" may mean very different things to different people.)

5weeatquince4moHi Tobias, Thank you for the comment. Yes very glad for CLR ect and all the s-risk research. An interesting thing I noted when reading through your recent comment [https://forum.effectivealtruism.org/posts/t5tP9iqHyKomMXqRn/what-are-novel-major-insights-from-longtermist-macrostrategy?commentId=GyKS3yKaamke28kLb] is that all 3 of the examples of progress involve a broadening of EA, expanding horizons, pushing back on the idea that we need to be focusing right now on AI risk now. They suggest that to date the community has perhaps gone too quickly gone towards a specific case area (AI / immediate x-risk mitigation) rather than continued to explored. I don’t really know what to make of that. Do you examples weaken the point I am making or strengthen it? Is this evidence that useful research is happening or is this evidence that we as a community under-invests in exploration? Maybe there is no universal answer to this question and it depends on the individual reader and how your examples affects their current assumptions and priors about the world.
5Tobias_Baumann3moYeah, I would perhaps say that the community has historically been too narrowly focused on a small number of causes. But I think this has been improving for a while, and we're now close to the right balance. (There is also a risk of being too broad, by calling too many causes important and not prioritising enough.)
3MichaelA4moThe post Tobias was commenting on requested "novel major" insights specifically. This guarantees that the examples provided would be ones that broadened EA, expanded its horizons, and pushed back on whatever priorities EA had before 2015. So I don't think we should read anything into the fact that a high proportion of the examples were of that kind, rather than e.g. refinements of existing ideas or object-level work within particular cause areas (since the question excluded such things). (That said, I do think that the number and nature of examples we can come up with in answering that question is relevant to how useful further cause prioritisation research would be. In particular, the fact that commenters came up with some examples rather than 0 examples seems to be evidence that some cause prioritisation research occurred and was useful over the last 5 years. And the fact they came up with relatively few examples is evidence that relatively little such research occurred or was useful. And this could perhaps inform our predictions about the future.)

Quick reaction:

I. I did spent a considerable amount of time thinking about prioritisation (broadly understood)

My experience so far is

  • some of the foundations / low hanging sensible fruits were discovered
  • when moving beyond that, I often run into questions which are some sort of "crucial consideration" for prioritisation research, but the research/understanding is often just not there.
  • often work on these "gaps" seems more interesting and tractable than trying to do some sort of "lets try to ignore this gap and move on" move

f... (read more)

I think you did a really good job nailing the emotional tenor of this post and I think it's great.

Thanks for writing this post! :-)

Two points:

i. On how we think about cause prioritization, and what comes before

2. Consideration of different views and ethics and how this affects what causes might be most important.

It’s not quite clear to me what this means. But it seems related to a broader point that I think is generally under-appreciated, or at least rarely acknowledged, namely that cause prioritization is highly value relative.

The causes and interventions that are optimal relative to one value system are unlikely to be optimal relative to anoth... (read more)

3MichaelA3moThis post [https://forum.effectivealtruism.org/posts/oovy5XXdCL3TPwgLE/a-case-for-strategy-research-what-it-is-and-why-we-need-more] - which I found interesting and useful - feels relevant in relation to your first point. A relevant excerpt: (I added two line breaks and changed where the diagram was, compared to the original text.) (That post was written on behalf of my former employer, but not by me, and before I was aware of them.)

I think there needs to be much better research into how to make complex decisions despite high uncertainty. There is a whole field of decision making under deep uncertainty (or knightian uncertainty) used in policy design, military decision making and climate science but rarely discussed in EA.

I think GPI is doing research on this, under cluelessness. See, for example:

... (read more)

Great post! I laid down a variety of comments and suggestions within your post using hypothes.is. If you want to check it out (you need to install the browser ad-in and get a free account to see these.

I prefer to comment within the text rather than here at the bottom, cutting and pasting quotes. Anyone else here tried hypothes.is?

(By the way, I'm an academic economist. I don't have any stake in hypothes.is. I just like it.)

I think the EA animal space is going beyond RCTs out of necessity, since RCTs have been hard to come by other than for diet change interventions (although their quality was previously quite poor, but better recently). Humane League Labs is researching the causal effects of corporate campaigns from observational data.

And you've already pointed out OPIS and the Happier Lives Institute, but HLI was incubated by Charity Entrepeneurship, which I think is generally looking beyond RCTs. They just put out their next round of recommended charities to incubate.

I fully agree with this!

"it doesn’t feel like the EA community has thought much about policy. For example there is a huge focus on AI policy, but the justification for this is weak. Even if you fully believe the longtermist arguments that top programmers should work on AI alignment, it does not immediately follow that good policy people can have more long term impact in AI policy compared to policy on resilience, macroeconomics, institution design, nuclear non-proliferation, climate change, democracy promotion, political polarisation, etc, etc."

>I like the idea of building "resilience" instead of going after specific causes.

That's almost exactly the approach we took in ALLFED, treating the more likely GCR and Xrisk scenarios as a "basket of risks"...
... and then looking at how to build resilience and recovery capacity for all of them, with an initial focus on recovering food supply.
We now have more than 20 EA volunteers at ALLFED, in a range of disciplines from engineering to history, so clearly this makes sense to people.

>For instance, if we spend all of our atten... (read more)

Ideas coming through my mind, not too well refined:

Reading this post, I came to think of this old joke:

A police officer sees a drunken man intently searching the ground near a lamppost and asks him the goal of his quest. The inebriate replies that he is looking for his car keys, and the officer helps for a few minutes without success then he asks whether the man is certain that he dropped the keys near the lamppost.
“No,” is the reply, “I lost the keys somewhere across the street.” “Why look here?” asks the su
... (read more)

Thank you very much for writing this up. However, I am not sure I understand your point, the things you are referring to in:

3. Policy and beyond – not happening – 2/10. Are you referring to your explanation within the subsection on The Parliament? Then, this would make sense for me.

2weeatquince4moYes that is correct. I have made some edits to clarify.