Elie, the CEO and co-founder of GiveWell, discusses his organization’s latest research, his views on economic growth, and what he’s changed his mind on lately (among other topics).

Elie co-founded GiveWell in 2007 alongside Holden Karnofsky. He also helps to set the strategy and oversee the work of Open Philanthropy, and chairs EA Funds' Global Health and Development Fund.

Before launching GiveWell, Elie graduated from Columbia University with a degree in religion and spent several years in the hedge fund industry.

Below is a transcript of a Q&A with Elie, which we’ve lightly edited for clarity. You can also watch it on YouTube and read it on effectivealtruism.org.

The Talk

Nathan Labenz [Interviewer]: Elie Hassenfeld, welcome to EA Global Virtual.

Elie: Thank you. I'm so excited to be here.

Nathan: It's great to have you. Obviously, we’re not talking under the ideal circumstances [given the COVID-19 outbreak], but we appreciate your making time to record an AMA [“ask me anything” session]. I think our audience is going to be very interested to hear directly from you.

Let's jump right into it. Just before this talk, we saw a video of a prerecorded talk by Neil Buddy Shah, who I understand recently joined as GiveWell's managing director. I know he hasn't started yet, but what will his work entail?

Also, we're looking for an update on GiveWell's recently announced research agenda. GiveWell announced that it's going to be scaling up research and focusing on some different policy interventions, including public health regulation around things like lead paint and tobacco, as well as potentially helping governments better implement social programs and improve governance generally. So there’s a new research agenda and new leadership. Tell us about it.

Elie: Yeah. To answer your good question, we're super excited about Buddy joining. Buddy is awesome. The reason he's coming on board is GiveWell has done a lot and had a huge impact in our first 13 years of existence. And the big question that we're trying to make progress on is: How do we increase our impact even more to get from where we are today to where we want to be in the future?

Buddy brings two things to the table. First, he's just an exceptional person. We've worked really closely together for the last five years while he's been running IDinsight. He founded or co-founded, and then ran, a successful, large research organization. And as GiveWell tries to take its operation to the next level, Buddy is going to be so helpful in getting us there.

But more concretely, GiveWell has been an organization that has been relatively internally focused in its first 13 years. And what I mean by that is a lot of the work we do focuses on reviewing academic papers, talking to charities, and then reviewing their monitoring and evaluation. And we think there are a lot of opportunities for GiveWell to increase its impact by broadening our external engagement with the global health and development community. We're already fairly looped in, but Buddy's primarily going to be focused on building those relationships and looking for opportunities. And I think those opportunities could come in a lot of forms. They could come from us learning from others about what they're doing in ways that help us recommend funding more effectively. But we also might have the opportunity to find ways that we could use the research that we do to support or even help improve the giving that some of the largest institutional funders, including governments, do worldwide.

When Buddy comes on, we think he's going to help by starting to explore some of those opportunities and hopefully [uncovering] one of those new areas that we can invest in to try and increase the overall value of the funds that we're able to direct. So that's one area.

Another area is what you just mentioned: some of the new [policy interventions] we're looking at. Again, this just comes back to GiveWell's evolution through our history. When we got started, Holden Karnofsky, who's my co-founder, and I, didn't know anything about charity. We didn't know anything about international development. And so we focused where data was most easily available — in programs that were relatively short-term and relatively quantifiable. And those programs and our top charities have had a huge impact. But the question is whether, by relaxing some of those restrictions, we can find opportunities that are even higher-impact.

We think one of the areas where we might be able to find more cost-effective or higher-value causes is in public health regulation — essentially, trying to support the passage of laws that improve public health.

These could be opportunities like lead regulation. We regulate lead paint heavily in high-income countries. Often, those regulations aren’t on the books in low-income countries. The same is true for tobacco [regulations]. And actually, one of the areas that I think we're most likely to be involved in — it's at the top of our list right now — is alcohol policy. Alcohol has a massive burden of disease globally, but philanthropically, it's very neglected. There are very few funders focused on it as a priority. And so the combination of a big public health burden and its relative neglect gives us some indication that it's a promising place to look.

Nathan: That's really interesting. And it leads into the next question. There's been a lot of debate in the EA community recently — and I probably would say in the broader sphere of development economics as a whole — about the impact of RCT-based [randomized controlled trial-based] approaches on trying to figure out what works and what doesn't. The RCT approach does tend to focus on relatively small-scale improvements to individual health and income, because obviously, to do an RCT, you must have a control group and a treatment group.

But some are arguing, I think more and more, that maybe there’s a need to increase focus on things that you might call “big structural change.” Typically, when people say that, they're thinking about broader economic development within a country — [targeting the reasons that some countries are rich and others aren’t] They might identify some middle ground — not necessarily huge structural change, but more pointed reforms that a country might be able to implement.

How is GiveWell thinking about that debate? I'm sure you're aware of it. Is this something that you're deliberately trying to move up the scale, so to speak? And how do you feel about the difficulties inherent in attempting to quantify [structural change]?

Elie: Yeah, it's a really great question. It's something that we do think about a lot. First off, I want to question the premise of the question a little bit, because I think sometimes the GiveWell top charities can get pigeonholed as small-scale and focused only on individual impact. And the programs that we support — malaria programs that deliver nets, seasonal malaria chemoprevention, deworming programs that treat hundreds of millions of children for parasitic infections — are massive public health programs that reach hundreds of millions of people globally.

And now that we're living in particularly unsettling times, we’re experiencing the fear in high-income countries of an unchecked infectious disease that we don't have the ability to effectively prevent and treat. And so I think these massive public health programs probably should do better on someone's scale of the potential for their long-term, large-scale impact. That's sad [that it has taken a pandemic to help people in high-income countries recognize the importance of mitigating infectious diseases elsewhere].

I think the argument that economic growth is ultimately extraordinarily important is one we take very seriously. [The economist] Lant Pritchett is often pointed to as the person making this argument very vigorously and publicly. He's someone we've talked to a handful of times. We take what he says into account. And this question of whether there are opportunities for us to accelerate economic growth is something that we intend to get to at some point in the future. It's on GiveWell's agenda, and we've started with public health regulation largely for the reason you stated, which is we think it's an easier way for us to move into the less measurable and more policy-driven parts of philanthropic giving. We think it's ultimately the right place for GiveWell to start — and hopefully not the right place for us to end.

Nathan: Cool. That anticipates the next question pretty well. I know GiveWell is very careful about going on the record with positions if you're not very confident about them, so feel free to pass on this question if you would like to. But we have a question about an interesting topic in development economics. It involves the hypothesis that we should think about things that developed nations actually did on their path to development as being somehow more proven, and therefore things that we should try to promote in developing countries.

For example, reducing trade barriers might seem to be more promising than investing in microcredit, because today's rich countries typically were more pro-trade in their history than others at the time. And they didn't seem to have microcredit institutions. That’s just one example.

How much credibility, if you have an opinion on this at all, would you give to the argument that things that rich countries did in history are things that developing countries today should try to emulate?

Elie: Yeah, we put a lot of weight on what's worked historically at GiveWell. So the argument really resonates with me personally, but also with our approach to what we do. Everything we do is focused on the track record of each intervention and charity. Even when GiveWell began the work that ultimately became Open Philanthropy, a big part of that was our “history of philanthropy” project, where we were trying to understand what had worked historically.

So the basic argument here is one that I think makes a lot of sense. I don't know enough about what has worked historically to comment thoughtfully on that. But I do know that even in the United States, in the early 20th century, malaria and hookworm were common. And early programs existed to eliminate malaria, and the Rockefeller Foundation led a hookworm eradication campaign.

I think those programs happen for good reason: A sick population is unable to do a lot of things that you need for a functioning economy. But all that said, I think this is an interesting question. I'd like us to take it seriously as we think about what's likely to work best, especially as GiveWell goes through this process of expanding the focus of our work from the types of interventions that are more easily measurable on a multiyear timeframe to those for which it's so difficult to get feedback in the near term that you have to lean heavily on the best case you can make, based on what's worked historically.

Nathan: Thank you. I should also give credit where it's due on these questions. They are coming from the AMA post in the EA Forum. So I don't want to make it seem as if I'm coming up with all of them. They were sourced from the community, and these are what we thought were the best of the bunch.

The next one focuses on the topic of gender inequality. Obviously, it's a big area of focus for a lot of development organizations. It seems, at least based on this question, that GiveWell hasn't really done much in the space of gender equality, reproductive health, or other related issues over time. Is that something that GiveWell has thought much about? Is that something that GiveWell would potentially consider as an intrinsic good beyond its contribution to general development? And what are your broad thoughts on the topic of women's empowerment?

Elie: GiveWell is broadly consequentialist in the giving that we do. We're focused on the direct impact that [our giving] has on the world. And so we take a utilitarian perspective rather than a perspective that considers the philosophical value of, say, justice, or of helping the least well-off. That's always been the frame that we've taken. And so causes that focus on equality, per se, are not ones that we've prioritized historically for that reason.

The way in which we could potentially treat that differently is via treating gender equality as an intrinsic value. It could be an instrumental value that leads somewhere else, and that would fit neatly within our framework. But it's not something that we've directly looked into before.

Within the broader framework of whether or not we would treat it as an intrinsic value, we certainly could. It has been a major challenge for us to think about how to weigh different good outcomes that charities achieve. Some charities improve health or save lives; other charities improve incomes or improve subjective well-being. And when we've looked at how other policymakers and funders have dealt with these issues, we haven't been satisfied with their approaches. And so we've tried to address this challenge directly via what we've called moral weights, where we try to explicitly compare the good achieved via two different charitable outcomes. So we want to [be able to say] how we value some amount of income increased by a charity against [a life saved by a] charity.

These are things where we don't have the right answers. There aren't any right answers. And our approach to trying to address these questions has evolved over time from just taking the median of what our GiveWell staff say — which is obviously problematic, but was the best that we felt we could do [early on] — to last year, [when we funded] a survey that IDinsight carried out to hear from beneficiaries about what they valued. This approach is still evolving internally; we've now designated a group of people on our team to really focus on these questions, both to figure out and assign weights for various outcomes, and also to decide which outcomes deserve to have intrinsic weight. And so we would essentially put this question to that group and have them spend some time thinking about it, sharing [their thoughts] with the team, and then sharing that with the general public. And so it's certainly something that could happen down the line.

Reproductive health is an area that we've specifically looked into. We just haven't yet found interventions that we see as competitive with our top charities on an impact basis. That's why we haven't recommended funding there. But it’s certainly within the scope of things that we would consider, and even look into, as we search for promising new organizations in 2020.

Nathan: Thank you. Next question: You're the chair of the Global Health and Development Fund, which is unusual among the EA Funds in that it almost always gives out larger grants — typically $500,000 or more. The other EA Funds tend to give out smaller grants in the range of $10,000 to $50,000 each. So tell us about your rationale for focusing on larger grants with that fund.

Elie: I think there are a few reasons that's happening. The first and most obvious one is that the work that we're doing to find opportunities for that fund is coming out of the research that GiveWell's already doing. We don't have a separate research team looking for opportunities for the fund specifically; GiveWell's research team is primarily focused on opportunities that serve GiveWell's needs. Those needs are finding really cost-effective places to put very large amounts of money. And so we tend to focus on the opportunities that are really big. I think that rationale is good, partially because our top charities do exceptional and very cost-effective work. I think that their ability to deliver programs at scale creates very cost-effective opportunities that are hard to find elsewhere.

Even as we have explored smaller giving opportunities, we've often found it very difficult to identify those that we really believe will end up being more cost-effective, over time, than our most cost-effective top charities. I do think that there's potentially an opportunity for someone who's looking for these small things to be good at finding them. There are other funders in the global health and development space that are looking for those opportunities and directing funding to them — groups like Ashoka, Endeavor, and the Skoll Foundation.

And so I think the global health and development funding infrastructure is more developed than the funding infrastructure for some of the other causes that are most salient to members of the effective altruism community. I also suspect that there's less of a need for that funding in global health and development than there might be elsewhere, though I'm not confident in that. And I certainly believe it's very possible that someone could identify really great small giving opportunities that we're missing because we're just not prioritizing looking for them.

Nathan: Do you know, off the top of your head, the rate at which a new charity is competitive with your top charities? Obviously, you have a certain selection bias. But of those that you find worthwhile enough to look into, how often does one threaten, if not fully crack into, that top echelon?

Elie: These numbers are a little bit off, but we started the year with a list of 50 or so opportunities that we thought would be potentially competitive with our top charities. That group of 50 [represents organizations that] really could compete for the last dollar that we would allocate or recommend this year.

In many ways, we now think of ourselves as a group that's responsible for allocating $100 million-plus in funding [each year] to opportunities in global health and development. Our primary focus every year is trying to improve the quality of that allocation, which means finding things that make the last dollar we give more cost-effective than it otherwise would be.

Right now, we're about three months into the year. And maybe there's a handful of things that have the potential to be better than the Against Malaria Foundation, the Malaria Consortium’s Seasonal Malaria Chemoprevention Program, some of the Deworm the World deworming programs that are very cost-effective, and some of Helen Keller International’s vitamin A supplementation programs. But those programs are just so exceptionally cost-effective, and collectively have so much room for more funding, that it's extremely rare [to find new charities that can compete for that funding].

I think the rate is extremely low primarily because we're not interested in diversifying for the sake of diversification. We're interested in trying to find the things to give to that maximize well-being to the greatest extent. And the current opportunities that we have are the best we've been able to find in the last 13 years. And those [charities] are just incredibly good at what they do.

Nathan: That actually connects to another question that I had queued up for a little bit later, but I want to ask it now: How do you go about your research process? It's striking to me that you’re only two and a half months into the year, and it sounds like you've already whittled the list down quite a lot. And I know you're extremely thorough in what you publish, so how do you know when to stop and when to keep going? What was the threshold for determining that as many as, say, 90% of the 50 charities you entered the year with have already been filtered out? If I have that right, what does that decision making process look like? Is it really going to be the case that the next nine-plus months is going to be focused solely on those last five prospects and then trying to qualify them fully for the list? Is that how it works?

Elie: You asked several questions. I'll try to answer a few of them; feel free to come back with others if I'm not fully addressing what's on your mind.

At every moment, we see our scarcest resource as our research time. And so we only want to spend research time investigating an opportunity when we think we get a good return on that time. We think of return on that time as a function of the likelihood that it changes some future funding decision that we make. So we might look into something very briefly and then figure out that it's extremely unlikely that it'll end up being a very cost-effective giving opportunity. The underlying analysis for that decision will be shared among the team that's working on it, and it will then eventually be shared with the wider team.

But the idea there is to maximally interrogate those ideas to see how well they're holding up under scrutiny before we drop something. And when we drop something, we're not dropping it forever. We're essentially deprioritizing it because we think, at the moment, that there's something else that is potentially even more cost-effective and could affect our funding to an even greater extent down the line. That's how we think about it.

The next nine months is not just about trying to qualify an opportunity for the list. For a long time, GiveWell was very focused on the top charities list as the only thing that we produced. It was essentially synonymous with GiveWell. Now, GiveWell makes a lot of funding recommendations to Open Philanthropy, to the Global Health and Development Fund, and sometimes even to other donors [for what we call] “incubation grants.” And so there's a lot of work that we're doing on an ongoing basis that's not just accruing toward the end of the year. Our work is a combination of seeing whether things should be on the [top charities] list, looking for other opportunities to make grants that are not accruing to the list itself, and then continuing to build our pipeline for the future across all of the different research domains that we focus on.

Nathan: Yeah, that's a fascinating approach. And I really like your framework for return on your research time, and even just the notion that research time is your scarcest resource. I think that’s a good insight.

Another question is related to GiveWell as a whole. This question starts by complimenting you and noting that GiveWell seems to be a very well-run organization. You're noted for your transparency, your prolific output of research, and your clear communication that always indicates what's up to date, what your level of confidence is in the position that you're taking, and so on. So great job on all of that. What tips on organization management and research strategy would you share with the rest of the community that aspires maybe to be a little bit more like GiveWell in their operation?

Elie: First of all, that's very kind. I really appreciate that someone would say that. It's also nice to hear because it's not how I feel day-to-day about GiveWell. I feel I'm mostly focused on all of the things that we're doing wrong and need to do better. It feels like we've made huge strides even in the last few years to fix things; it’s hard to believe now that they were broken for so long. So it's nice to hear from the outside that someone has that impression of us. It's certainly something we strive for: to run an effective organization.

One of the things we've tried to do well — I'm not sure we have — is put a very heavy priority on building great relationships between managers and direct reports. I think those relationships have helped us. Especially recently, staff routinely get really great feedback about things that aren't working. [That has helped us] build a cohesive organization where, frankly, we're able to have [a level of] internal transparency that would be difficult to achieve if people didn't feel as comfortable with one another. We've also been very focused, especially recently, on being incredibly judicious about the roles we hire for and the people whom we put into those roles.

I think that we sometimes wonder whether our recruitment process is too stringent and too long — whether we're losing people whom we wouldn't otherwise, [though we mostly see that people who apply are willing to go through the process]. But what [our stringent process leads to] is that often, after the recruitment process, we're in a position to feel pretty good about our ability to forecast what someone will be able to do on the job. That has helped us make really good decisions about whom to hire. This is something we've put a lot of energy into over the last three or four years. It's something we do a lot better now than we used to.

When I think about what makes GiveWell great today, and what is going to make GiveWell succeed in the future, it's just this team of exceptional people — exceptionally talented, exceptionally committed, exceptionally good about their core values and commitment to what GiveWell's trying to do — that's going to make us succeed. Ultimately, the organization is just the people. And so building good relationships with them and being really thoughtful about which roles you hire for and how you select those people has been really important to us. It's the thing we obsess over all the time, and it's probably the number-one thing that I advise others to do.

Nathan: On that note, I was going to actually end with this question, but now it seems like a good time to ask: What are the roles that you are looking to hire for today? I'm sure many people here watching will be interested to learn about that from you. And are there any particular backgrounds or areas of expertise that you are specifically looking for right now?

Elie: Yeah, there are two big areas. I'd say our number-one priority is senior research capacity. And what I mean by that are researchers who can really help drive GiveWell's future research agenda. Half an hour ago, at the beginning of the call, we started talking about new areas and questions that GiveWell wants to look into. And the biggest reason we're not moving as quickly on that as we'd like is that we just don't have the research capacity that we want. We need researchers who are both familiar with quantitative, empirical research methods and focused on getting things done in the real world.

We've had a lot of struggles, because sometimes we meet people who are incredibly smart and great economists, but they want to go all the way to the end to answer a really complicated technical problem. And then other times we meet what I’ll call the “management-consultant mindset” person who's really practical, but doesn't want to get into some of the weird questions that we need to deal with. We're really looking for the people who can marry those two qualities and bring both the technical or quantitative knowledge, as well as the real-world application.

Getting back to something that we were talking about earlier, we really focus on the return on our time invested. And so we need people who know when to go deep — and can keep going deep when necessary — but also know when they've gone far enough.

I see these research hires as the number-one priority for GiveWell right now. If we can hire a great additional group of researchers to work with the team we have, I've no doubt that GiveWell will continue to succeed in its research work into the future. We really need to [hire that group], because without it, we just won't be able to make the progress that we hope to.

The other big area of focus for GiveWell is outreach. For a long time, GiveWell primarily focused on research. We didn't invest a lot of resources in trying to raise money for our recommended charities. And we were just incredibly fortunate that people found us and talked about us. Donors came to us. And we were mostly reactive in our outreach. And now, we're trying to focus on building out more of that team to raise more money for the groups we recommend.

And so we're also now hiring for a position called a gift officer. That's someone who works with major donors to help them understand GiveWell's work and its research, with the goal, over the long run, of building those relationships to the point that donors give more money to our recommended charities because they feel confident about [our work]. We think that job will be exciting for someone who really likes explaining things to people and building relationships. Some development or sales experience could be really helpful, but it's not absolutely necessary. Some of the people we have on staff have essentially learned on the job and that [would work] well in that position.

Nathan: What does the portfolio look like for an individual in that role? Is it 10 people whom they work with, and how close are those relationships?

Elie: It varies based on the size of the donor. Broadly, it's about 100 people whom a gift officer would have in their portfolio. They’re people I have probably met at least once, but don't meet in-person every single year. We try to stay in touch with them over time to ensure that both the donor understands what we're working on and we understand which parts of our work the donor is most interested in. People are interested in all different aspects of the work, and there are different types of information that they find most useful. Knowing enough about them to know what to share can really help us give them the type of information that's most helpful to them in settling on their giving decisions.

Nathan: Yeah, that's a really fascinating approach. How many people do you have in that role today?

Elie: Two people full-time in that role. And if we grow in the way we want, that function could easily become a team of five people in the next few years. Currently, that position is just severely understaffed relative to the number of donors we have in our community.

Nathan: Yeah, I would imagine you could go well beyond five over time. That reminds me of another thing that I was thinking about in anticipating this conversation: GiveWell's advertising. I hear spots read by Ezra Klein on his podcast and other podcasts from time to time.

I know that time may be donated to you because Ezra Klein identifies as an effective altruist. We were actually going to speak at EA Global about his book until, obviously, the event went in another direction. But if that is [advertising] space that you're actually paying for, how do you think about the ROI [return on investment] for advertising?

Elie: We're paying for that space. One of the things that we're focused on this year is scaling up our paid marketing — or, I should say, we’re at least planning to focus on that. I think with everything that's happening right now [related to COVID-19], we're going to step back and reassess what makes the most sense. I expect we're still going to move forward to some extent with that. But the world has changed a lot in the last month. And so we're thinking about what we want to do there.

We pay for those spots and we measure the ROI based on the dollars that we put in relative to the dollars that we're able to track going to our recommended charities as a result of that advertising. In 2019, we just saw an exceptional ROI from those advertising spots on podcasts. So we spent about $150,000 total, and we saw about $500,000 in total donations to recommended charities that we were to track as a direct result of those ads. We know that over time, most dollars that come in during the first year continue into subsequent years. And so even that is an understatement of the return on investment that we ultimately expect to get.

Now, the thing that we know is we have to be careful about where we advertise. We got an excellent return on investment with The Ezra Klein Show in 2019. There have also been podcasts where we spent a lot and haven't received that same return. And so the big question for us is trying to figure out where it makes sense to test, and then where it makes sense to scale, because our goal is to maximize the dollars that go to our recommendations. And if we see opportunities to take $100,000 and either put it into a top charity or use it to immediately drive $500,000 to that top charity, it's pretty obvious that we want to be advertising to drive more money to our recommendations.

Nathan: That's interesting. Obviously, you're only able to track so much, especially when you're doing a pseudo-broadcast medium like a podcast. Do you have a theory or a sense for what percentage is trackable versus what’s untrackable?

Elie: We really don't know. That's a really tough question.

Nathan: Yeah, nobody in marketing does. But I thought you might at least have a theory.

Elie: No. We wrote a blog post a few years ago where I made some absolutely crazy estimates about what it might be. By crazy, I mean entirely unfounded. But we really don't know. We asked some donors to put up matching funding for our podcast advertising last year, both because it created an incentive for people to give and made it easier to track; in order to receive the match, people had to give via a specific URL. And so we think that helped us.

But there's no question that when we spend roughly $150,000, we see $500,000 in donations as the floor, because there's the lifetime value of the donor to consider, and on average, the dollars are going to keep coming in.

We've also consistently seen in GiveWell's history that a large portion of the money we move comes from very large donors who hear about us in the mass media, spend a long time doing research, and then ultimately [donate] a few years down the line. And so we expect that there are donations that will end up coming through from someone who heard about us on The Ezra Klein Show, or on podcasts in general, and whom we don’t know about yet.

What we're trying to do is find those opportunities. Right now, it's a no-brainer to keep advertising, if we can do it in the right places, and to scale up advertising to the appropriate level. And we just have to figure out the right way to do it, in the right places, and at the right level so that we don't underspend or overspend, which is obviously going to be a challenging call for the marketing team to make.

Nathan: Very interesting. Well, not too much time remaining, but I want to ask one timely question — which, again, you can potentially pass on if you don't have an opinion — and then we'll do a few classic EA questions as well.

First, the timely question: Obviously, we're doing this remotely because of what is now a global pandemic. Do you have any thoughts about effective giving opportunities that respond to crises in general and this crisis in particular? And is there anything that you think could be on the level of your top charities, potentially?

Elie: We're not sure yet. It's a good question that we all are asking internally. We are looking into opportunities to respond directly, especially in low-income countries. It seems as if in high-income countries, the pandemic is the government's highest priority. A lot of money is going toward trying to address it. We suspect that we can’t add value by directing funding there. But I've also heard from a lot of people in low-income countries — both at the charities we recommend and people who run health facilities — how frightened they are of what might happen in a context like Sub-Saharan Africa, where the health infrastructure is relatively poor.

It’s going to be an even bigger problem there than it is in high-income countries — places like the U.S. and the Bay Area where I live — because the infrastructure is so much worse in Sub-Saharan Africa. And so we're trying to look into what responses could be at both the charity-specific level, where they're looped into health systems and health workers, but potentially also more broadly. It's something that we don't know the answer to yet because it's too early to say.

Nathan: Yeah. Do you feel pressure to do something? You are obviously very rigorous in your approach, but is there a sense of “We can't sit this one out”? Or would you be comfortable just saying, “Hey, we spent some time researching this and we can't find anything, so we're just going to hold the line on our top charities”?

Elie: We're always driven by trying to find the opportunities that have the highest impact. And we have sat out a lot of disasters in the past, even when people are asking questions about where to give. That’s because, frankly, we just don't think we have a good answer for them that will really help them. We have a blog post with high-level tips (6 tips on disaster relief giving), which we share in disasters. But that's about the best we've been able to do. And so we do sit it out.

If in this case we end up concluding that boring, everyday malaria control is a better use of donor funding than addressing COVID-19, that's something that we're going to say, and that's something that we're going to do. It's a position that we all feel very fortunate about — that we have a donor community that ultimately really supports our core values. Ultimately, they want to see us trying to direct money where we think it will improve people's lives the most, rather than responding to the cause of the day. And so we're in a position where we can really hold the line on doing the research that we think is most valuable.

Nathan: Yeah, that's an admirable relationship to have been able to create with your stakeholders.

Now for a few classic EA questions. First, zooming out and thinking about the EA movement as a whole, I think everybody's familiar with the broad trajectory when it started. It wasn’t synonymous with GiveWell, but very much focused on the same core things as GiveWell. It focused on global poverty, and health and development issues.

In more recent years, it has broadened. We now have a broader portfolio of causes, a lot of which are more speculative, and a lot of smaller grants that allow us to try to find the next hits. What do you personally think of that shift? How do you feel the movement is evolving? And how has that affected GiveWell as an organization?

Elie: The effective altruism community focuses on causes that are just so important: humanity's long-term future, animal welfare, and building this movement. Those are all exceptionally valuable. And so in my position at GiveWell, I'm really happy that those other organizations and groups working in those cause areas all exist, because it enables GiveWell to focus in the area where I think we can do the most given our position, our organizational culture, and our comparative advantage. The fact that there are people, donors, and organizations focused on, for example, trying to improve farm animal welfare as much as possible, enables me to say, “The reason GiveWell is not thinking about that today is that there are others doing that work. And we don't think we have that much to add.”

I think that overall, it's been very beneficial to GiveWell. I remember what things were like in August 2007, which was my first day full-time at GiveWell. There was no community, there were no people who were interested in asking the questions that the community's asking. There certainly was no EA Global being held where people could ask questions about global health and development during a global pandemic. I think all of this has created a very rich and fulfilling community, which I'm very proud to be a part of. And personally, it's really nice to be able to have my day job at GiveWell and then go listen to Toby Ord on the 80,000 Hours Podcast talk about his new book. The amount of intellectual energy that's going into all of these activities [is great to see and experience].

Nathan: Yeah, I agree wholeheartedly with that. My last question is a classic two-parter. First of all, what is something that you think a typical EA Global attendee might be getting wrong either about global health and development in general, or about GiveWell's work in particular — a misconception that you’d like the chance to correct? And then part two: What is something that you have personally changed your mind about recently, whether that's specific to GiveWell’s work or otherwise?

Elie: Maybe one of the things that I worry about most with folks in the EA community is that people may think that because staff at global health and development organizations don't speak the EA language, that indicates they are not smart, competent, and thoughtful. My experience has really been the opposite. I’d like to say, very explicitly, that those folks are often incredibly knowledgeable and incredibly thoughtful, but they just don't speak in the same way that the folks in the EA community speak about expected value and priors. And I think that can lead to real miscommunication. It can lead to an erroneous assumption that those organizations don't know what they're doing.

I worry sometimes in the EA community that people sometimes think they need to start things on their own because no one's doing the work that needs to be done, when in fact, there are a lot of people out there doing some of this work — especially in global health and development. They just don't speak the same language. I think it can also lead to problems in taking up an organization's time if you're trying to get them to speak in the EA movement’s language and they normally speak another way. It's really important to be cognizant of the way that the global health development community can communicate and try to meet them where they are and hear them out. I think that can really help all of us in the EA community better understand where they're coming from.

[In terms of misconceptions] about GiveWell specifically, I think GiveWell is still often seen as the group that is primarily focused only on short-term randomized interventions, and that those are the only ones we'll consider. And this is a big part of what GiveWell does. I think it's always going to be a big part of what GiveWell does. But GiveWell has already slowly expanded into supporting things that are much harder to quantify. And so I hope that people see that in our work we've really moved into looking for things that I think most EAs would consider higher-risk and less empirically grounded than conventional GiveWell top charity stuff.

Nathan: And how about something that you have changed your mind about recently?

Elie: All right. I knew that one was coming. Maybe I'll give a quick one about GiveWell and I can also give a quick personal one, just in case it's helpful. I feel like I change my mind on an ongoing basis as I learn things. One of the things that I changed my mind about at GiveWell recently was in really thinking about how we should engage with our donor community.

In GiveWell's first few years — especially in those first few years when folks donated to GiveWell's top charities — they were really interested in all of the intricacies and details of our research methodology. And that meant that they'd want to get on the phone and argue about whether the Against Malaria Foundation's approach to monitoring and evaluation was sufficient to be confident in their impact, or if it was insufficient and we made a mistake in recommending them.

I had those sorts of debates constantly with our donors. And that led us to develop an approach to outreach that was very research-focused. It meant that we were, for a long time, engaging with our donor community with the expectation that that is what they wanted. And it wasn't until last year or recently, I don't remember exactly when, that we determined that our donor community had really changed. The people I talked to in 2009, 2010, 2011 — they were early adopters. And they were folks who were very individually engaged in the details of what we were doing in a way that's different from a lot of the people who find us now.

Our donors are great, but many who give to GiveWell charities now are not spending 50 hours every year reading through all of our research to decide where to give. We still have some donors like that, and they’re great too, but a lot of them have spent some time kicking the tires. They trust GiveWell. They want to rely on our recommendations and they need a different type of engagement from us than, “Hey, I'm a researcher and I'm happy to debate with you about the intricacies of our research.” And so we shifted that a little bit to try and meet donors where they are and make sure that we're giving them the information that they want — which sometimes is not a debate about the research methodology.

Nathan: Yeah, you built a brand. And people, as you said, can trust it. The imprimatur that you've established over a decade has put you in a position now where people don't feel the need to go into that same level of depth that they used to.

Elie: Yeah, I think the brand is there. We still hear from people who, when they first become donors, are looking at the research and reading a footnote or two. That's very common. But they're often not going further than the first few hours of research. So maybe our next iteration is getting to the point where the folks don't even need to go to the footnote because we have the support that makes that possible. I think a challenge we need to overcome is that it still takes people a lot of time to ultimately feel totally confident about giving a lot of money through us, and we want to keep making that path even easier for potential donors.

Nathan: Awesome. Well, this has been a fascinating, nearly hour-long conversation at this point. So we thank you for your leadership in the EA movement, and we thank you also for your time today joining us for EA Global Virtual.

Elie: Yeah, thank you. And I just want to say thank you to you, Nathan, and everyone who posted questions on the EA Forum and is watching this right now. We're living in a very unsettling time, and I really appreciate that people are staying engaged with this, notwithstanding the situation we're going through.