Topic Contributions

Comments

FTX/CEA - show us your numbers!

P.S. I've also just seen Joan's write-up of the Focus University groups in the comments below, which suggests that there is already some decent self-evaluation, experimentation and feedback loops happening as part of these programmes' designs. So it is very possible that there is a good amount of this going on that I (as a very casual observer) am just not aware of!

FTX/CEA - show us your numbers!

I completely agree that it is far easier to suggest an analysis than to execute one! I personally won't have the capacity to do this in the next 12-18 months, but would be happy to give feedback on a proposal and/or the research as it develops if someone else is willing and able to take up the mantle. 

I do think that this analysis is more likely to be done (and in a high quality way) if it was either done by, commissioned by, or executed with significant buy-in from CEA and other key stakeholders involved in community building and running local groups. This is partly a case of helping source data etc, but also gives important incentives for someone to do this research. If I had lots of free time over the next 6 months, I would only take this on if I was fairly confident that the people in charge of making decisions would value this research. One model would be for someone to write up a short proposal for the analysis and take it to the decision makers; another would be for the decision-makers to commission it (my guess is that this demand-driven approach is more likely to result in a well-funded, high quality study). 

To be clear, I massively appreciate the work that many, many people (at CEA and many other orgs) do and have done on community building and professionalising the running of groups (sorry if the tone of my original comment was implicitly critical). I think such work is very likely very valuable. I also think the hits-based model is the correct one as we ramp up spending and that not all expenditure should be thoroughly evaluated. But in cases where it seems very likely that we'll keep doing the same type of activity for many years and spend comparatively large resources on it (e.g. support for groups), it makes sense to bake self-evaluation into the design of programmes, to help improve their design in the future.

FTX/CEA - show us your numbers!

It's bugged me for a while that EA has ~13 years of community building efforts but (AFAIK) not much by way of "strong" evidence of the impact of various types of community building / outreach, in particular local/student groups. I'd like to see more by way of baking self-evaluation into the design of community building efforts, and think we'd be in a much better epistemic place if this was at the forefront of efforts to professionalise community building efforts 5+ years ago. 

By "strong" I mean a serious attempt at causal evaluation using experimental or quasi-experimental methods - i.e. not necessarily RCTs where these aren't practical (though it would be great to see some of these where they are!), but some sort of "difference in difference" style analysis, or before-after comparisons. For example, how do groups' key performance stats (e.g. EA's 'produced', donors, money moved, people going on to EA jobs) compare in the year(s) before vs after getting a full/part time salaried group organiser? Possibly some of this already exists either privately or publicly and the relevant people know where to look (I haven't looked hard, sorry!). E.g. I remember GWWC putting together a fundraising prospectus in 2015 which estimated various counterfactual scenarios. Have there been serious self-evaluations since ? (Sincere apologies if I've missed them or could find them easily - this is a genuine question!)

In terms of what I'd like to see more of with respect to self-evaluation, and tentatively think we could have done better on this over the last 5+ years: 

  • When new initiatives are launched, serious consideration should be paid to how to get high quality evidence of the impact of those initiatives, which aspects of them work best. 
    • E.g. with the recent scale-up of funding for EA groups and hiring or full time coordinators, it would be great if some sort of small-scale A/B test could be run and/or a phased-in introduction. E.g. you could take the top 30-40 universities/groups that we'd ideally have professional outreach at and randomly select half of them to start a (possibly phased-in) programme of professional group leading at the start of 2022-23, and another half at the start of 2023-24.
    • Possibly this is already happening and I don't know - apologies if so! (I've had one very brief conversation with someone involved which suggested that it isn't being approached like this)
    • One objection is that this would delay likely-valuable outreach and is hard to do well. This is true, but it builds knowledge for the future and I wish we'd done more of this 5+ years ago so we'd be more confident in the effectiveness of the increased expenditure today and ideally have a better idea what type of campus support is most effective!
  • I would love to see 1-4 people with strong quant / social science / impact evaluation skills work for ~6-12 months to do a retrospective evaluation of the evidence of the last ~13 years of movement-building efforts, especially support to local groups. They would need the support of people and organisations that led these efforts, to share data on expenditure and key outcomes. Even if lots of this relied on observational data, my guess is that distilling the information from various groups / efforts would be very valuable in understanding their effectiveness.
What music do you find most inspires you to use your resources (effectively) to help others?

I just came across 'Ordinary Pleasure' by Toro y Moi, the lyrics of the chorus: 

Maximize all the pleasure
Even with all this weather
Nothing can make it better
Maximize all the pleasure
Maximize all the pleasure
Even with all this weather
Nothing can make it better
Maximize all the pleasure

Economics PhD application support - become a mentee!

Yes please - that would be great! The easiest way is probably to just fill out the form and just clearly note somewhere that you'd be happy to be a mentor. Thanks!

Economics PhD application support - become a mentee!

Yep, there's no requirement to have EA-related research ideas yet (and I'd expect that to be the case for a decent number of EA-interested and aligned people who participate in the mentoring programme)

An introduction to global priorities research for economists

Thanks a lot for sharing the syllabus, David, and for posting guidelines about using it. I think and hope this will serve as a really useful reference for people interested in pursuing (a career in) economics GPR. As you note, this is a bit broader than GPI's current research focus (which is fairly narrowly focused on longtermism and associated questions for the time being), but I think there is valuable GPR to be done in these other areas too. As you also note, GPI is currently refreshing its research agenda to account for some of the exploration research we've done in economics over the last ~18 months - hopefully we'll have a new and improved version out in the next 2-3 months.

Why I've come to think global priorities research is even more important than I thought

I just wanted to explicitly add to this post that valuable GPR can, does, and should happen outside of an academic setting. I think this is implied in this post (e.g. the mention of OpenPhil and the link to the GPR roles on the 80k website), but is not quite explicit, so I just wanted to flag it. Researchers outside of academia face a different set of incentives to academics, and can sometimes have more freedom to work on questions that are more practically relevant but less 'publishable' in academic journals. The point is made quite nicely on the 80k GPR page here: https://80000hours.org/problem-profiles/global-priorities-research/#what-are-some-top-career-options-within-this-area

"That said, we expect that other centres will be established over the coming years, and you could also pursue this research in other academic positions.

One downside of academia, however, is that you need to work on topics that are publishable, and these are often not those that are most relevant to real decisions. This means it’s also important to have researchers working elsewhere on more practical questions."

Personally, I think/hope the field of GPR will develop in a similar way to 'impact evaluation' in development economics over the last ~20 years -- i.e. significant progress has been made in academic research (including some of the more important methodological or foundational advances), but there has also been a lot of valuable non-academic impact evaluation research (including lots that is more directly relevant for decision-makers).

The case of the missing cause prioritisation research

This is a great post - thanks a lot for writing it. I work at GPI, so want to add a bit of context on a couple of points, and add some of my own thoughts. Standard disclaimer that these are my personal views and not those of GPI though. 


First, on GPI's research agenda, and our progress in econ:

"(One economics student told me that when reading the GPI research agenda, the economics parts read like it was written by philosophers. Maybe this contributes to the lack of headway on their economics research plans.)"

I think this is accurate and a reflection of how the research agenda was written and has evolved. For what it's worth, we're currently working on refreshing the research agenda to reflect some of the 'exploration research' we've done in economics in the past ~18 months - we should have an updated version in the next few months. More generally, we've had very little econ research capacity to date beyond pre-doctoral researchers (very junior in academic terms). This will improve very shortly -- as Phil notes in a previous comment, we've hired two postdocs to start in the next month -- but as others have noted, high quality academic work is hard and takes quite a lot of time, so this may not result in a step change in actionable econ research coming out of GPI in the short run, which leads on to my second comment... 


Second, on theories of change - your point D1 is really important. We've actively discussed various 'theories of change' internally at GPI and how these should affect our strategy. A decent part of this discussion depends on what others are doing in EA and how we think GPI fits into the overall EA movement portfolio. Even within the (relatively narrow) scope of doing academic GP research in econ and philosophy, possible theories of change for GPI include (but are not limited to!) prioritising building up academic credibility for long-run influence, prioritising research that is more actionable for EAs/philanthropists and policymakers, prioritising influencing policymakers / the general public, or prioritising influencing the next generation through higher education. These are not mutually exclusive, but placing different emphasis on one or the other may imply different strategy. We are still very young, and so far we have mostly been focused on laying foundations for the first of these, and have so far made much more progress on this in philosophy than econ, though I expect things will evolve in the next few years. Personally, I don't think we'll be able to effectively target all of the possible theories of change, and I'd love to see more people and groups working on these. 

One for the World: update after 6 months of our first staff member

Thanks, Aaron. Just to add to Steve's response below:

1) We think that part of the reason for the large increase in people pledging ~1% of their income this academic year is due to (a) better training and messaging (largely because we now have a full time staff member doing this), and (b) our improved donation platform. (b) has allowed us to set different defaults for different chapters, and for donors to set their start date further in the future, so it coincides with graduation. Before that, our donation defaults were targeted at MBAs, so undergrads in particular would log on, see a massive default relative to their expected salary, and pledge something significantly lower. Juniors and sophomores would also be unable to pledge far enough in the future to start giving at graduation and so would often put in a 'symbolic' $10 pledge, which partly explains why the average donation (as percentage of average graduation income) was pretty low before this year.

2) Our donation data indicates that the power of defaults is pretty strong among our members (a large mass of people give the default amount on the sign up page). We plan on experimenting with changing these default options (e.g. having 2% and/or 5% as well as 1%) and testing whether this makes a difference. At first we will just do this on the online platform, but we'll also consider randomly selecting some chapters to have messaging about higher percentages in future years, and again testing if this makes a difference.

Load More