Comments
Charity Entrepreneurship is a research and training program that incubates multiple high-impact charities annually. Founded in June 2018 by Joey Savoie and myself, Karolina Sarek, it builds on our experience in research and direct work on global health, animal advocacy, and other causes, including founding and incubating two GiveWell-incubated charities, Charity Science Health and Fortify Health. In this post, I summarize the results of the research we conducted in 2019 and our plans for 2020.
In 2019, we published 66 pieces of research on animal advocacy. Some examples of our work include:
As a result of our research, we identified 9 promising ideas for new charities. Our top 5 recommended charities for 2019 were:
In 2019, our Incubation Program graduated 13 alumni, who went on to launch six new charities. Five of the new charities implement interventions highlighted as promising by our research program:
How was this outcome accomplished, and how did these charities come into existence? Read our recent update.
This year we are expanding our scope. We plan to research and incubate organizations in four areas: animal welfare, mental health and happiness, global health policy, and family planning.
We generated and considered ~300 ideas in each cause area. First, we spent 20 minutes evaluating each idea using four methodologies: cost-effective analysis, weighted factor model, informed consideration and expert view (an idea sort report). Then, we investigated the top ~10% of ideas for 2 hours (an idea prioritization report). Next, we selected the ~8 most promising ideas in each cause for in-depth research. Further details of the research process will be described in the next section.
Mental health directly influences happiness and can fill large gaps in both high- and low-income countries. The evidence base also looks promising despite limited historical work on determining the most cost-effective interventions. Read more about why we’ve chosen to prioritize this area.
Based on our idea sort and idea prioritization report, the most promising ideas in this area seem to be:
Farmed and wild animals are victims of huge suffering and extreme neglect. There appear to be many promising ways to massively improve animal lives. Now seems like a uniquely good time to found new animal organizations.
Based on our idea sort and idea prioritization report, the most promising ideas in this area seem to be:
Family planning has a range of positive effects on health, wealth, empowerment, and more. The area looks highly promising and is growing dramatically but has not yet applied the same methodologies as other areas of global health. Read more about why we prioritize this area.
Based on our idea sort and idea prioritization report, the most promising ideas in this area seem to be:
Leveraging government resources can make a significant, large-scale impact. Health policies can be evidence-based and extremely cost-effective.
Based on our idea sort and idea prioritization report the most promising ideas in this area seem to be:
Our research process focuses on where to found new charities; as such, it differs from a research process focused on producing new knowledge. Knowing the principles of the process helps readers understand how we formed our conclusions and enables greater reasoning transparency.
Our research process incorporates elements that are well established in some fields but uncommon in others. This is partly because of the unique goals of our research (i.e. finding new areas for impactful charities to be launched) and partly because we incorporate lessons and methodologies from other fields of research, primarily global health and medical science. Below is a quick overview of some of the key elements.
Iterative depth: We research the same ideas in multiple rounds of iterative depth, with each round looking at fewer ideas than the last. Our goal is to narrow down our option space from a very large number of ideas to a more workable number for deeper reports.
Systematic: The goal of our research is to compare ideas for a possible charity to found. To keep comparisons consistent our methodology is uniform across all the different ideas. This results in reports that consider similar factors and questions in a similar way across different interventions. This systematic approach is commonly used in other charity evaluations and encouraged in other fields.
Cluster approach: Comparing different intervention ideas is complex. We are not confident that a single methodology could narrow down the field, in part due to epistemic modesty. To increase the robustness of our conclusions, we prefer instead to look at ideas using multiple independent methodologies and see which ideas perform well on a number of them (more information here). Each methodology is commonly used in most fields of research but they are rarely combined into a single conclusion.
Decision relevant: Our research is highly specialized and focused. Sometimes cross-cutting research is needed to allow comparison between different ideas, but all our research aims to be directly useful to our endline goal of getting new charities started. This level of focus on target practical outcomes is rare in the research world, but is necessary to our goal of generating more charity ideas with minimal time spent on non-charity idea related concepts.
Transparency: We highly value transparency in conclusions, mistakes, reasoning, and process. Expressing reasoning transparency is crucial for informing future charity founders and others who can asses our work and provide us with feedback. We view this as key to improving our charity recommendations and processes in the long term.
Following those principles, we designed a new research process. The goal of our research is to have one to four recommended charities for each of our four cause areas in time for the 2020 Incubation Program. Each step of the process is documented below, specifying its goal, characteristics, and methods. Hyperlinks offer further information about certain steps.
Our process is as follows:
STEP 1: GENERATING IDEAS
For each cause area we generate around 300 ideas for interventions that could be implemented as new high-impact charities.
4 cause areas x 25 hours = 100 hours spent
STEP 2: SORTING IDEAS (20-MINUTE SORT)
At this stage, each idea goes through a four-stage process, with each stage lasting approximately five minutes per idea. This results in all the ideas being sorted from most to least promising.
Cost-effectiveness Analysis: Comparing the cost of a given action to the amount of good it achieves.
Expert View: Conducting interviews with experts broadly informed on each cause area.
Weighted Factor Model: Generating pre-set criteria and weightings, and then evaluating how a possible option scores on each of these.
Informed Consideration: Using a non-predetermined research process, and taking into account soft forms of evidence and information that do not fit into other research methodology categories.
4 cause areas x 300 ideas x 20 minutes = 400 hours spent
STEP 3: PRIORITIZING IDEAS (2-HOUR SORT)
The most promising 10-30% of ideas are selected for each cause area, and 2-hour reports are created for them. At this stage, the sorting methodology varies depending on the cause. (An explanation of why we picked which methodology for which cause can be found here.
Animal Advocacy: Weighted Factor Model Report
Health and Development Policy: Informed Consideration Report
Mental Health and Happiness: Cost-effectiveness Analysis Report
Family Planning: Expert View Report
At the end of the process, all ideas are again sorted from most to least promising.
4 cause areas x 30 ideas x 2 hours = 240 hours spent
STEP 4: CHOOSING TOP IDEAS (80-HOUR REPORT)
At this stage, we create 80-hour reports for the top 10 or so ideas from the idea prioritization report. These reports state our prior views and repeat in greater depth the 4-stage process from STEP 2, examining the idea through the lens of cost-effective analysis, expert views, weighted factor model, and informed consideration. Our goal is to develop a top charity ideas list consisting of a limited number of interventions.
Depending on various factors, we predict spending at least:
4 cause areas x 10 top ideas x 80 hours = 3200 hours spent
STEP 5: CHARITY IMPLEMENTATION REPORTS
We prepare implementation reports for the top 9 charity ideas that we recommend. They will help charity founders kick-start new organizations immediately after the program.
Animal Advocacy: 4 top ideas
Health and Development Policy: 2 top ideas
Mental Health and Happiness: 2 top ideas
Family Planning: 1 top idea
Each stage of the research may be supported by:
_________________________________________________
We will publish all reports open access on our blog to provide other researchers and organizations with useful data. We may also experiment on cross-posting our report on the EA Forum.
If you’re interested in our work, please consider subscribing to our newsletter. If you would like to talk about our work, feel free to contact me at [email protected]. If you'd like to support our work, you can donate to us here.
Really impressed by your work so far, thanks for sharing this.
I'm curious about how you are using multiple researchers for this. Most steps can be done in parallel, but I wonder- how much do you rely on multiple views on the same analysis, and how do you go about it?
Also, is there anything that the EA community can do to assist the research process? If so, what could be the most valuable? (I'm interested specifically in small volunteer research projects that non-experts can take without your explicit direction, perhaps reviewing reports or rechecking ideas that did not successfully passed through the funnel)
Hey Edo, I'm glad to hear that you find our work useful.
We have one lead researcher for each cause, responsible for conducting comprehensive research in their area; this way, they become experts in their respective fields. But we also want to capitalize on the fact that we are one of only a few organizations conducting research in multiple causes. We’re in a unique spot to learn and cross-apply methodologies and practices from other causes, as Neil Buddy Shah illustrates. Animal advocacy can cross-apply from global health research e.g. a comprehensive system to grade the quality of evidence. In turn, global health can learn from animal advocacy e.g. how to answer questions when there is little information, or when the evidence-base is low. For this reason, after the initial draft of a report is completed, it is peer-reviewed by a researcher from a different cause. On top of that, we have a senior staff member whose work is dedicated to thoroughly reviewing the reports. He looks for contradictory research; challenges crucial assumptions; double-checks key inputs in the CEA; verifies that the strength of evidence has been adequately expressed in the report based on its source; etc. At the end, I analyze the conclusions of the report. So for example, I consider whether any crucial considerations have been missed; if the evidence is strong enough to warrant the conclusion; and if equal rigor has been applied across different charity ideas. We also engage external research reviewers and experts in the field.
I'm always looking to improve our systems, so I'm open to suggestions on how we can do things better.
Thanks for this question and for facilitating this research group! It seems like a fascinating project, and I cannot wait to see updates from it.
Researching marginal ideas on our priority list would be most valuable (ideally using the same process so it is comparable). Ideas that almost made it to our priority list probably have the highest odds of being better than the idea we recommend, so researching them might change what charities will be started. To get more granular, it would be really helpful to conduct crucial consideration research that may determine whether an intervention merits deeper research. As an example, here are the first ideas that didn’t quite make the list for each cause:
1. Mental health and subjective well-being: Addressing mundane, suboptimal happiness through conditional cash transfers for using gratitude journals
2. Animal welfare: Developing and advocating for pre-hatch sexing to reduce the suffering of male chicks
3. Family planning: Informing parents and girls about future economic opportunities
4. Health and Development Policy: Improving health systems through community monitoring of health problems (e.g. through scorecards, planning meetings, etc.; regional comparison/competition for outcomes-focused government)
You can read about each of these possible interventions in more detail in the linked Idea Prioritization reports.