Karolina is co-founder and Director of Research at Charity Entrepreneurship. There, she creates a research agenda and processes and leads the research team, aiming to find and compare the most evidence-based, cost-effective and neglected interventions in multiple cause areas. She also trains and mentors Incubation Program participants in starting high-impact charities. She also serves as a Fund Manager at the EA Animal Welfare Fund, and as a board member and consultant for various nonprofits and think tanks.

Before Charity Entrepreneurship, she co-founded an organization to improve the impact of nonprofits and social enterprises; worked on measurement and evaluation; and was a researcher for IBM and the Jagiellonian University (JU). At the age of 22, she became a university teaching fellow, lecturing at JU’s Faculty of Mathematics and Computer Science.


Charity Entrepreneurship’s 2020 research plans
Really impressed by your work so far, thanks for sharing this. 

Hey Edo, I'm glad to hear that you find our work useful.

I'm curious about how you are using multiple researchers for this. Most steps can be done in parallel, but I wonder- how much do you rely on multiple views on the same analysis, and how do you go about it? 

We have one lead researcher for each cause, responsible for conducting comprehensive research in their area; this way, they become experts in their respective fields. But we also want to capitalize on the fact that we are one of only a few organizations conducting research in multiple causes. We’re in a unique spot to learn and cross-apply methodologies and practices from other causes, as Neil Buddy Shah illustrates. Animal advocacy can cross-apply from global health research e.g. a comprehensive system to grade the quality of evidence. In turn, global health can learn from animal advocacy e.g. how to answer questions when there is little information, or when the evidence-base is low. For this reason, after the initial draft of a report is completed, it is peer-reviewed by a researcher from a different cause. On top of that, we have a senior staff member whose work is dedicated to thoroughly reviewing the reports. He looks for contradictory research; challenges crucial assumptions; double-checks key inputs in the CEA; verifies that the strength of evidence has been adequately expressed in the report based on its source; etc. At the end, I analyze the conclusions of the report. So for example, I consider whether any crucial considerations have been missed; if the evidence is strong enough to warrant the conclusion; and if equal rigor has been applied across different charity ideas. We also engage external research reviewers and experts in the field.

I'm always looking to improve our systems, so I'm open to suggestions on how we can do things better.

Charity Entrepreneurship’s 2020 research plans
Also, is there anything that the EA community can do to assist the research process? If so, what could be the most valuable?

Thanks for this question and for facilitating this research group! It seems like a fascinating project, and I cannot wait to see updates from it.

Researching marginal ideas on our priority list would be most valuable (ideally using the same process so it is comparable). Ideas that almost made it to our priority list probably have the highest odds of being better than the idea we recommend, so researching them might change what charities will be started. To get more granular, it would be really helpful to conduct crucial consideration research that may determine whether an intervention merits deeper research. As an example, here are the first ideas that didn’t quite make the list for each cause:
1. Mental health and subjective well-being: Addressing mundane, suboptimal happiness through conditional cash transfers for using gratitude journals

2. Animal welfare: Developing and advocating for pre-hatch sexing to reduce the suffering of male chicks
3. Family planning: Informing parents and girls about future economic opportunities

4. Health and Development Policy: Improving health systems through community monitoring of health problems (e.g. through scorecards, planning meetings, etc.; regional comparison/competition for outcomes-focused government)

You can read about each of these possible interventions in more detail in the linked Idea Prioritization reports.

EA Animal Welfare Fund is looking for applications until the 6th of February

Hey Abraham!
Thanks for asking for clarification.

One can submit an application until the end of the 6th of February (24:00 GMT).

What should Founders Pledge research?

Thanks for asking this question. I support and follow the approach of asking relevant people in the space for input to a research agenda. I am happy to see that other organizations are also doing it.

Meta-thoughts on how to approach this selection task would also be handy.

Your question inspired me to write a short post on a methodology of systematically integrating stakeholders' and decision-makers' input into the research agenda. You might find this meta-methodology helpful.

Out of the areas you mention, I'd be very interested in the following:
Animal product alternatives 6/10

Pain relief in developing countries 6/10
Improving science 9/10

Ideas not included on your list:
GiveWell recently published its list of areas they are planning to explore. I think some of them might be of interest to donors focused on improving the welfare of the current generation of humans and high-income countries’ problems.

  • Tobacco, alcohol, and sugar control
  • Air pollution regulation
  • Micronutrient fortification and biofortification
  • Improving government program selection
  • Improving government implementation
  • Immigration reform
  • Mosquito gene drives advocacy and research
  • Mental health (interventions comparison)
  • Sleep quality improvement

As you know, GW’s research is very diligent. Consequently, it takes a long time to finalize. I would be interested in having preliminary research conducted by other organizations.

Regarding donors focused on animal welfare:

  • Producers’ outreach, for example,. providing subsidization for farmers interested in higher-welfare farming
  • CRISPR-based gene drives to address wild animals’ suffering
  • WAS intervention comparison
  • Affecting law and law enforcement focused on welfare improvements for chicken and fish in Asia
  • Insects’ welfare, intervention comparison, for example, reduction of the production of silk, painkillers for insects used in research, etc.

I am currently working on CE’s agenda for the next year in the area of global poverty/health, animal advocacy, and mental health. I will be able to list more areas and research questions worth investigating that CE cannot cover this year at the end of September. I am narrowing down a list of research ideas from 400 ideas (in three cases). Let me know if you are interested in hearing more about it.

Did corporate campaigns in the US have any counterfactual impact? A quantitative model

Thanks Saulius for pointing this out. We spent a considerable amount of time trying to find a more relevant number, but that was the closes proxy we've found. If you have any other source we could use, that would increase the accuracy of the estimate.

Did corporate campaigns in the US have any counterfactual impact? A quantitative model

When analyzing 35 Independent Pieces of Evidence for Why New Corporate Campaigns Might (or Might Not) Work, I’ve found some commonalities:

  1. Follow-through rate is lower in the case of companies that affect the highest number of hens.
  2. Financial situation: When companies broke or delayed the commitment, they blamed the recession or claimed that there were insufficient consumer demand and lack of funds for making the necessary changes on farms.
  3. ACE claimed that the success of the campaigns depends on the public perception of targeted issues. I have not fack-check that though.

Characteristics that don't correlate with follow-through:

  1. No correlation between the number of pledges and the % of cage-free eggs in a given country before the campaign. Meaning that campaigns do not seem to be simply riding already existing trends.

I wouldn't be surprised if those factors were somewhat predictive:

  1. If the pledge was made voluntary form the company or was forced upon by strong negative campaign.
  2. If a given company gave itself some wiggle room in the phrasing of their commitments, which they could later be used as a justification for breaking their commitments.
  3. What brand and public image they try to create
35 Independent Pieces of Evidence for Why New Corporate Campaigns Might (or Might Not) Work

Thanks for the suggestions! As we were discussing above, combining this estimate with a prior estimate using Bayes’ rule might be a solution here. Taking the uncertainty of the model into account, we indeed score this approach quite poorly when it comes to the evidence-base aspect of it. We have a different research template for approaches than the one you linked. I expect to publish the whole report on corporate outreach pretty soon.

35 Independent Pieces of Evidence for Why New Corporate Campaigns Might (or Might Not) Work

When it comes to the step between research questions and the probability distribution, full research, answering each question, can be seen in the full report. In the report, we also address some of the concerns you have with the judgement calls on each of the “qualitative” parameters.

Each update incorporates the weight we put on this factor, the directionality and strength. Those factors, again, rely on other information. With the example, you cited ”what ACE thinks makes an effective campaign” vs “probability that all companies defect in a Prisoner's Dilemma scenario". For example, ACE’s opinion on the importance of public support when launching corporate campaigns is formed based on the intervention report they have researched in November 2014, and as they currently claim “is not up to our current standards.”. The landscape has changed since then. As of recent, we can observe that there is a strong track record of successful corporate campaigns in countries where the society didn’t have sympathetic views toward animals (e.g. Lithuania or Japan). I think we can rely more and more on rigorous and generalizable conclusions from research on real-life examples and on the application of game theory to predict the behaviour of the companies.

I agree I wish we had enough time to flesh out the reasoning for each of the factors. Sadly, due to limited time we are constantly having to make trade-offs about whether we should put time into explaining the reasoning more deeply to the broader community vs discussing with the CE candidates vs researching more to get a deeper internal understanding. We generally plan on going deeply into these factors with the specific entrepreneurs looking to start this project or others, who are going to work/are working in the field in the near term, but not publish much more on the topic publicly after our full report.

35 Independent Pieces of Evidence for Why New Corporate Campaigns Might (or Might Not) Work

Thank you for the suggestion. I agree that we can’t extrapolate the conclusion about predicted follow-through rate based on what percentage of companies have followed through on the commitment so far. I looked at it again, and I think that if analyzed correctly it still provides valuable information, so I will leave it in the model, but I’ll move it to the section on updates based on qualitative data and change the values based on the information below.
I think that a good proxy for the information is whether the top 20 biggest companies are making progress and will eventually switch is if they responded to EggTrack. We can break it down to:
* Companies that passed their deadline:
Whole Foods (2004) - 100% follow-through
Costco (2018) - as of July 2017, 78% and 100% converted to liquid
*Those that didn’t respond to EggTrack are Walmart, Albertsons, McDonald’s, Target, Sysco, ALDI, Burger King, Tim Hortons, Southeastern, and Wendy's.
* Those that did respond and reported progress: Kroger (21% as of 2017), Publix (50%), SUPERVALU (as of 2016, 12%)
* Those in 20 that I do not have information on US Foods, IGA, Inc., Associated Grocers of Florida.

I will estimate the value in this cell based on this information unless you have info that could fill in the gaps.

I generally think that with all very uncertain estimates, whatever the result, it should be only treated as a cautious update and be combined with prior estimates of value.

As a meta comment, I think I’m less concerned about an error in one of the parameters than you seem to be because of the different goals of the research. My goal is to reach broadly good conclusions about which intervention should be executed from a given list of options given a limited amount of time, rather than get the right answer to a specific question, even if it takes me an extremely large amount of time. I think that using cluster approach is superior in such cases. If you are using cluster approach, the more perspectives you take into account the lower the odds of your decision being wrong, and so I trade other aspects (eg. number of interventions compared in a given time frame and how accurate a single estimate need to be) differently. One contradicting factor also cannot overpower the whole decision, etc. A completely different method should be used when we are trying to have as accurate beliefs about the world as possible vs getting to a good decision.

Top Charity Ideas 2019 - Charity Entrepreneurship

When evaluating cost-effectiveness of interventions or charities, GiveWell only looks at how the action affects the most important metric that the charity is trying to adress. For example, when analysing the cost-effectiveness of SMS vaccine reminders, they only take into account the effect on the vaccination rate, but not on breastfeeding rates, which is also promoted by the intervention. We look at the effect on multiple disperse metrics, including health effects, reduced birth rate, woman empowerment, effect on animal welfare and the environment, etc. Additionally, we have not yet determined that condom distribution is going to be the intervention the charity is going to pursue. We are also considering SMS for reproductive health, community reproductive education, advance provision of EC etc.

Load More