When you start a charity, you clearly want to do good directly — but you also have an opportunity for other forms of impact, from building the effective altruism (EA) movement to generating knowledge that can be shared with many other people. In this talk, Katriel Friedman, CEO of Charity Science Health, discusses his organization’s experiences with the indirect benefits of charity entrepreneurship.

Below is a transcript of Katriel’s talk, which we’ve lightly edited for clarity. You can also watch it on YouTube or read it on effectivealtruism.org.

The Talk

This talk is going to be a bit of a sequel to a talk that [my co-founder] Joey Savoie gave at EA Global London in 2018. 

 

Joey’s talk was about charity entrepreneurship as a promising career path for EAs. He argued that startup charities [could] have greater potential for impact than the most promising existing interventions, mainly because so many ideas are untried or under-tried — and very few organizations have been founded based on our rigorous search for the most promising interventions. 

He also spoke about multiple indirect or pass-through benefits of starting a charity. For example, they attract money, attention, and passion to really promising issue areas [even for other organizations in the area]. Or perhaps, through trial and error, they teach us important lessons about what works and what doesn't work. 

Over the last year, Eric Housen [Housen is the COO of Charity Science Health] and I have been gradually taking the helm of Charity Science Health, living in India, and steering the organization through what you might think of as its early teenage years. Ninety-nine percent of the time, we focus on our direct impact: How we can send more memorable messages to mothers and save more children from deadly, painful diseases like measles, pneumonia, and diarrhea? 

However, [in this talk] I’m stepping back to ask what we can learn about some of the indirect effects of charity entrepreneurship. 

 

I'll go through them broadly and share whether these benefits have materialized in our experience or not. 

First, to get us all on the same page, I’ll provide a quick overview of what Charity Science Health does and how we got to this point. I'll then break down some [indirect benefits of starting a charity] into three broad categories:

 

1. Ways that charity entrepreneurship might strengthen potential future projects;
 

2. Ways that charity entrepreneurship might restructure the funding environment to be favorable to highly effective projects; and 
 

3. How charity entrepreneurship might generally support building the EA movement.

Finally, I'll say a few words about how we can get involved as a community to support charity entrepreneurship through the Charity Science Network.

Overview of Charity Science Health

Charity Science Health (CHS) was born out of a research project that the founding team and an army of volunteers conducted. They set out to find the most promising interventions in global health that only needed a dedicated entrepreneurial team to take them forward. 

 

At the very top of the list was short message service (SMS) reminders for immunization, and so the Charity Science teams spun up CSH to implement the idea. 

The reasons SMS reminders were so promising are that (1) there is robust evidence for their impact and (2) they are extremely cheap to implement. 

 

We found six randomized controlled trials (RCTs) showing that the effect, on average, of receiving reminders was an increase in the immunization rate of about ten and a half percentage points. And to send our whole suite of reminders only costs us about six cents. 

This is a recipe for terrific cost-effectiveness. 

 

Indeed, GiveWell's preliminary cost analysis of the program found a cost per life saved of about $1,500. That's actually a bit conservative because it reflects the very bottom of the range of the studies that we evaluated, and is a higher cost per enrollment than what we have subsequently demonstrated with some of our partners. 

The core task for CSH is to enroll caregivers as cheaply and as scalably as possible. Therefore, we started out by listing enrollment strategies and trying them out, quickly piloting what would work, and picking the strategies that seemed to be the most promising. We've been gradually refining those and finding ways to make them more reliable, cheaper, and more scalable.

 

As we run out of great ideas for how to improve these strategies, we’re going to solidify our program and conduct an RCT. That will happen at some point in the next six months. If the results come back as strong as we're hoping, that will signal that it's time to approach GiveWell, ask to be considered for “top charity” status, and begin the work of scaling up across India.

We've learned that two strategies are the most viable.

The first is going door to door in slums and villages, trying to find as many mothers as we can and giving them the chance to sign up for the program. The second is tapping into health records kept on paper registers in health facilities or in digital databases by the state governments, and using those to directly enroll caregivers in our program. At this point, we’ve reached 250,000 families across six Indian states. 

 

Just in the last few weeks, we've made some exciting [progress] with state governments in India's federal system. It's the states that have primary responsibility for citizens’ health, and so these governments are either sharing data with us already or allowing us to go into health facilities, digitize records, and upload them into our system for sending messages. 

Hopefully, [that overview] will provide something of a foundation for what comes next.

How charity entrepreneurship strengthens future projects

The category of indirect benefits that I have the most to say about is how charity entrepreneurship strengthens other projects, so that's where I'll start.

Startup charities create networks that future projects in the same area can potentially tap into. We've seen this go both ways. 

 

At CSH, we have benefited greatly from networks set up by people like the Fortify Health team and the Good Food Institute. And we're also quite confident that the government relationships we’ve forged — the working relationships with state health officials, who are the mid-level officials likely to stick around over the long term — could be accessible and relevant to other teams working in the same area who have the same values and priorities. 

 

A lot of these really valuable connections are actually business relationships. We've found good contractors. And we’ve found a few teams that have been instrumental in shepherding along the process of working with the Bihar Health Department. Bihar is a state with three million births per year, which provides a sense of what the scale of India is like. Another person we’ve found has deep ties to everyone in the Indian government and is able to tell us anything we want to know about the initiatives people are working on. 

 

So, to me it seems clear that these networks are a benefit of doing startup charities.

 

Another benefit is that startup charities might be an uncommonly good training ground for EAs. This is a bit of a bold claim, but [I’ve repeatedly noticed that] there are skills that come with being an EA-aligned organization that would be hard to pick up working for other NGOs. 

 

For example, thinking about cost-effectiveness is just not that common in NGOs. Almost every conversation I have with another nonprofit consists of someone brainstorming ways we could make our program more expensive. It might have a bit more of an impact, but they want to add trainings, follow-ups, and awareness-raising.

I recently listened to a podcast with Chris Blattman, a University of Chicago professor in the Harris School of Public Policy [see this blog post for more context on Blattman’s views on cost-effectiveness].

He has decades more experience dealing with this topic. [I was gratified to hear him say] that this [problematic urge to increase programs for current beneficiaries] is universal among NGOs. At CSH, we try to resist the temptation to favor our current beneficiaries over our potential future beneficiaries [who might be reachable thanks to the current program’s low cost].

 

We have really structured processes for making decisions. Almost every time we make an important decision for the organization, we create a spreadsheet and slap a scoring function on it. We weigh every important criterion and do our best to quantify each one. 

 

One of the qualitative things I'd hate to quantify is how much faster Eric and I are growing as a result of working at this organization than we would have in our counterfactual roles. [Laughs.] But it seems extremely plausible to me that because we and our team members are in an organization without preset ways of working — and with strong ties to people writing and thinking a lot about rationality — it's a much more fertile learning environment than [organizations] we might have ended up in otherwise.

 

In addition to helping individuals learn skills, startup charities give future projects an opportunity to learn about what works and what doesn't. We like to think that this is in our organizational DNA. We are called Charity Science because we like to run experiments. And we've been celebrated publicly for [proactively] acknowledging mistakes.

 

The other thing to say on this topic is that we've certainly learned from examples of organizations like Fortify Health, which figured out how to address some legal concerns involving regulations of foreign contributions to NGOs in India. I think we were able to adopt [their method] it more quickly than they did [because they’d already done it]. That solved so many operational problems for us and really reduced our legal risk as an organization. [Meanwhile,] the lessons we've learned at CSH have become the backbone of the Charity Entrepreneurship incubation program. 

 

I think it's fair to say that this kind of learning is both significant and inevitable with startup charities. 

 

The final way that startup charities support future endeavors is by inspiring other founders to create them. This is something that has definitely happened in the case of CSH. 

 

Brendan [Eappen] and Nakita [Patel] have gone on the record acknowledging that the example of CSH played an important part in their decision to launch Fortify Health. Charities evidently beget other charities.

 

How charity entrepreneurship affects the funding environment

Charities can't exist for long without starting to have an effect on the funding environment. Where an organization gets its money is a big part of determining its net impact. Therefore, it's important to think about organizations that take their funding from sources with the same values and priorities as the EA community, or in the impact philanthropy space — as well as organizations or initiatives that are funded by donors with other values, priorities, and reasons for providing support. 

 

When it comes to donors, the core ambition of CSH and the other Charity Science Network charities is to improve the [quality of] EA donation opportunities available to donors. Obviously, I can't say yet whether we will accomplish this, but it’s important to us to be able to [increase the likelihood of this outcome]. That's why we're going to do an RCT. 

We also want our RCT to be accompanied by a pre-analysis plan. One of the pitfalls we're worried about is the growing consensus that promising interventions get good results at a small scale, but when they're scaled up, those results evaporate. 

 

To address that, we've been in frequent conversations with IDinsight and GiveWell. We've thought about setting a minimum size that we have to be before we evaluate [our impact]. I think what we're leaning toward right now is setting up rules to firewall our [work on studying our impact, so that it isn’t interfered with by] our leadership. We also might be able to get deceptively positive results by picking a geography that's favorable for us — one with low back-to-baseline immunization rates or high female literacy. [To try to avoid that], we're going to map out every place that we would want to reach at scale and pick something closer to the median [on immunization rate, literacy, and other relevant factors] as our target study area.

We also think carefully about monitoring. We want to make sure we don’t break the program as we begin scaling. We’ll measure immunization rates in our enrollment process and do phone surveys of a sample of people to make sure that they understand the message. We'll know if something breaks and we're no longer having the effect we [did previously]. At CSH, we have the freedom to make deliberate choices to ensure that effectiveness-minded donors know where we stand relative to our competitors.

 

That said, startup charities have a bigger net impact when they find donors who have other values and other motivations for supporting them [outside of effectiveness alone]. 

 

It's not yet clear whether we'll be able to do this. I think that we haven't had to do very much fundraising as a result of the generosity of this community and GiveWell. We’ve just begun embarking on that journey. We do have one piece of evidence: Even without giving any attention to fundraising, a foundation found us online and offered us a substantial grant over the course of three years.

 

An even further-reaching effect that charity startups could have on the funding environment is to catalyze the interest of funders in entirely new issue areas. 

 

In the case of CSH, this is probably a bit of a stretch. We work in immunization, and truthfully, the biggest global health donor has already gone all in on immunization. 

 

So frankly, [getting even more interest] is a bit of a tall order [for us]. 

 

How charity entrepreneurship might support building the EA movement

The final broad category is ways in which we can strengthen the EA movement. [My co-founder] Joey [Savoie] suggested that one way startup charities do this is that they create institutions that preserve the principles and values of the EA community — even as individuals [within those institutions experience] value drift or [leave to] focus on other projects. 

 

It's hard to say in the short term whether CSH is doing anything like this. I hope that CSH and the apartment from which four of us live and work continues to be a hub for EAs. 

 

I also hope that intense contact with the hardcore EAs of CSH causes our team members and our close contacts to begin taking these ideas more seriously. We certainly have staff who now read EA content and hadn't previously heard of the term “effective altruism.” But to be frank, we haven't had anyone pledge to give what they can or become a vegetarian, at least not to our knowledge. So, we’re still working on that.

 

Supporting charity entrepreneurship overall

Overall, I want to ask: Does any of this matter?

I would like to say that attracting non-EA funding will justify the whole purpose of these startup charities. I think it’s an open question whether the organizations that are really effective at impact will prove to be really effective at storytelling and fundraising outside of the EA community. We now have this growing crop of startup charities that are well on their way to having to deal with that. 

 

And so we'll be able to say much more about that in the future. What I can say is that we have seen really convincing evidence that building these networks is an important effect of startup charities. And we've seen these networks grow reliably wherever charity entrepreneurs are at work. This is what emerges to me as the biggest pass-through benefit.

 

What can we, as a community, do to promote charity entrepreneurship? CSH and Charity Entrepreneurship each have opportunities to accomplish more with the support and involvement of the community. 

 

CSH, for the first time in a while, has room for more funding. Over the course of the next 18 months, in order to continue pressing forward with an [impact evaluation] and our initiatives with the state governments in India, we will need about $460,000 beyond what we can expect to receive from GiveWell, as well as an academic partner to finance an RCT of our program.

If you have any leads on people who can help me understand the nonprofit fundraising environment or who might be interested in hearing more about this work, they would be enormously beneficial in helping save lives in India. 

If you yourself are interested in learning about starting a nonprofit, or have friends who would like to learn more about it, take a look at charityentrepreneurship.com and sign up for the mailing list. 

 

The incubation school is drawing a strong pool of candidates for their current cohort, so there's very likely to be another incubation program in the next year. 

If you'd like to talk to me, my email address is katriel@charityscience.com. 

Thanks very much. I'm really excited to hear your questions.

Moderator: Thanks for this talk. It’s an ambitious topic to try to take on. Someone in the audience has a question: If you think that some of the interventions that you'd be pushing for are as effective as they seem, is it ethical to run an RCT?

Katriel: This is an excellent question. It's a core part of the research ethics that they teach you when you're being trained to run an RCT — whether you can run them if you have equipoise (i.e., are certain that an intervention works). 

Now, that has become a bit of a joke in the impact evaluation space, I think because no one knows how to quantify the certainty that you need before you can can make this decision. And you can almost always make an argument that equipoise exists.

In the case of CSH, I think we really do have sufficient uncertainty for an RCT. This is a behavior change intervention. And I gave you a pretty favorable account of why we should be confident, but whenever you take a behavior change intervention to a new context, there are reasons that it might break down. People may not respond to it in the ways you expect. And that's why GiveWell has insisted that before they give serious consideration to CSH as a top charity they would want to see an RCT in the Indian context. 

[Generally], we focus on being implementers. I apologize if I gave [the impression] that this [talk represented] a far-ranging and balanced review. I really wanted to offer something that was more of a tentative report from the field. We generally take the view that we will let people like GiveWell make a clear-eyed assessment of what's valuable.

Moderator: One of the indirect value-adds you mentioned is growing a network. How might starting a charity in India create a useful network for somebody starting a charity elsewhere? Can you describe how that network functions?

Katriel: That's a great clarifying question. I was absolutely talking about a network that's helpful to other projects in India — working in the Maharashtra and Bihar states, for example. I think that it's reasonable when you're considering a startup charity to think very carefully about the resources that you'll be able to draw on — and whether you can draw on people like Varun Deshpande, the advisor to the Maharashtra chief minister. 

Varun Deshpande is the Indian presence for the Good Food Institute [and another EA Global speaker]. He is a phenomenal, charismatic presence and has a history of accomplishing amazing things like setting up a cellular agriculture research center. He has found someone who understands the EA community’s evidence-based way of thinking and who sits in a position of tremendous political influence in India. This advisor, when I first interacted with him, sent me an email that said, “What you're doing is terrific, and if you get stuck anywhere in this state or if anyone gets in your way, just let me know and I will unstick you.”

The fact that Varun and I have this close tie means that he's able to uncover people like that who will be accessible and relevant to me. That's pretty amazing.

Moderator: For somebody who is interested in going through the Charity Entrepreneurship school, what are some of the top things that they'd really like to see somebody start a charity on?

Katriel: I'm not qualified to answer that question. I really urge you to watch the charity entrepreneurship talk that Joey gave at EA London 2018 .

Moderator: If someone were to try to either integrate into the existing network or try to build a network of their own, what are some key pieces of advice that you would give them (assuming they have no background in this area)?

Katriel: This is a question from someone who's trying to develop a network that will be effective for charity entrepreneurs? That's how I'll interpret it. 

This is a really excellent question. One of the things that I've heard people commenting on (and noticed myself at EA Global this time) is that the community is becoming increasingly effective in its thinking about policy, and more tied to senior policymakers. So coming to EA Global is probably not a bad start. 

For me, the most transformative thing in my career was entering a public policy master’s program that had a lot of incredibly effective and thoughtful people. To me, the key benefit of that master's program was seeing how others organize their lives around the goal of having a better, bigger impact in the world. I got to meet them. And those people have spread out into the world and become the seeds of a really vital and far-reaching network.