I mostly agree. Though having worked in research, policy and now policy analysis, it's also the case that sectors can get stuck in established practices. Big-picture /blue sky thinking is of course also important. But I agree that the value of work to get the basics/details right is under recognised. So thumbs up for your post, except the framing of "boring"; one person's boring is another's wildly fascinating - who are we to judge. 

Thanks for sharing - it’s an interesting interview. My first reaction is that interdepartmental bureaucracy is quite a different beast to an evidence-to-policy process. I agree that splitting development policy/programmes across multiple government depts causes lots of problems and is generally to be avoided if possible (I’m thinking about the UK system but imagine the challenges are similar in the US and elsewhere).

Of course you do need some bureaucracy to facilitate evidence-to-policy too, but on the whole I think it’s absolutely worth the time. For public policy we should aim to make a small number of decisions really well. The idea a small efficient group who just know what to do and crack on is appealing; it’s a more heroic narrative than a careful weighing of the evidence. Though I can’t imagine the users of this forum need persuading of the importance of using evidence to do better than our intuitions and overcome our biases. 

Incidentally, I feel this kind of we-know-what-to-do-let’s-crack-on instinct is more acceptable in development policy than domestic, and in my view development policy would benefit from being much more considered. We cause a lot of chaos and harm to systems in LMICs in the way we offer development assistance, even through programmes that are supporting valuable services. I think all of the major GHI’s do great work, but all could benefit from substantial reforms. Though again, this is somewhat separate from the point about interdepartmental bureaucracy. 

Thanks for your comment and I agree. Modelling (even rigorous modelling) is just that, a model. It's a simplification of a more complex reality. We should not mistake the map for the territory, but equally, not using a map would be foolish. 

Greetings GiveWell Colleagues, 

Let me first re-emphasise that this blog is very much in a “yes and…” spirit. As I say in the blog, I believe EA is a positive influence on philanthropy and global development and has more potential to continue to shake things up for the better. 

Thank you for this detailed response and the clarification that some of your analysis and grant making is indeed context-specific; great to see, particularly as someone who cut their teeth on CEA of malaria interventions. My impression is still that context-differentiated analysis perhaps the exception rather than the rule within the EA space - i.e. including organisations beyond GiveWell and topics beyond global health. It doesn’t yet come out strongly in the back-and-forth in forums like this one nor in presentations at EA Global conferences and it’s usually not emphasised in recommendations of EA donors - perhaps including GiveWell if we look at the evidence in the Top Charities page (as you note). Solving this challenge is not straightforward of course and remains an issue for other donors in the development space. 

I don’t mean to suggest there are off-the-shelf lessons from HTA - on this or the other differences I highlight - that could be adopted directly by EA organisations. Equally, the field of HTA has been developing approaches for cost-effectiveness-based decision-making for several decades and may be fertile ground to explore for the development of EA prioritisation. 

Happy to keep in touch as useful.

Ok gotcha this time. Similar to some of the response to Jason below I would say that the assessment of value and the assessment of gaps etc are separate steps. We shouldn’t think about adopting a value framework because it allows us to find a practical niche. I suppose it does beg the question; whose value framework should we use? I don’t have that answer, but I do think that it links to the 4th difference in my blog; about institutionalisation and participation. 

Btw, you imply that the rest of the aid landscape uses this HTA-like approach to prioritisation.. Sadly this is certainly not the case! I worked in the UK Department for International Development for several years, in a team that was tasked with providing evidence to the rest of the organisation. We did our best but it was a long way from perfect. Indeed I do see a significant opportunity for EA movement and organisations to influence other donors to take a more systematic, evidence-informed approach. For me this absolutely does mean using cost-effectiveness - but it means using it not in a blunt way but as part of an appropriate process and so that - wherever possible - we work with, not around, local systems and institutions. 

Thanks Jason, appreciate your thoughts. 

Taking your points in order

1. Yes I essentially agree - though the other option available is to decide to work more through local institutions to prioritise and fund health services. Again yes this may be better done at scale. Though with the rate of growth in EA funding, I wouldn’t rule out a role for this kind of EA engagement. Or perhaps there is a role for EA as a kind of catalytic funder, investing in frameworks/platforms for more effective aid policy from states and major multilaterals. 
2. I think you mean that if there are gaps for highly cost-effective services that aren’t being delivered by other donors (or indeed the national health system, we should remember that even in the very poorest countries, national institutions are still likely to be the main healthcare funder and provider, not the donors) then it makes sense for EA to fill those gaps. I think gap-filling and prioritisation are separate steps. With the additional criteria we’re still trying to assess value and therefore in a sense a certain kind of more broadly conceived assessment of cost-effectiveness. For example, if intervention A has a better cost per DALY than intervention B but equity considerations mean the HTA committee strongly prefers intervention B, then B is rightly considered a higher priority. We could then look at the practical issues of gaps, organisations that can deliver etc. 
3. On specificity of recommendations, yes that’s fair. As you say this is just a blog so perhaps for future work. I also considered working out a clearer taxonomy of what is being prioritised (causes/programmes/interventions/charities etc). But this one was on the long side for a blog as it is

Hi Ben 

First off - I see HTA as a kind of process wrapper for cost-effectiveness analysis. So in a sense It's CEA+, rather than an alternative. 

I'm not sure I followed you points about moral uncertainty and emulation of pluralism. Would you try again to say what you mean? 

On Justin's blog, CGD welcomes - even encourages - a diversity of views within its staff. I don't agree with everything in this blog. For example HTA (which as I say, encompasses CEA) is precisely the tool used by many countries in price negotiation. I think there will be a follow up CGD blog with the opposing view, but I would say that CEA/HTA is now adopted by a large and increasing number of countries to inform health policy. If we're saying it's good enough domestic policy but not aid policy, we need to be really clear why.

Thanks,

Thanks Ryan!

I see you like the capability approach - I'm a fan too

Thanks for this considered response Jason. 

I would summarise your overall message as “we’re not ready for this yet” and I would partly agree. But EA community and orgs could be part of developing the solution and influencing other donors. We shouldn't consider EA orgs as separate from bilateral, multilateral or other philanthropic donors. At this point, EA is not an insignificant voice in the global health ecosystem. Also the shift doesn't need to happen for the sector as a whole, it could be country by country and some countries probably are ready for such an approach. We're discussing it with a couple. 

To some of your more specific points:

I disagree that fungibility is the biggest challenge. Excepting cases of corruption, this simply means the country is effectively choosing the spend on other priorities. Indeed some folks in CGD have argued against the marginal aid approach precisely because they favour the fungible quality of current donations. This isn’t my view but it speaks to an opinion I do share, which is that, in many cases, countries should be given more control over healthcare priority setting.

Of course a common concern is whether country institutions are good faith actors and have the necessary capabilities to adopt a marginal aid approach. This will differ between countries and can be considered on a case by case basis - and will change over time (inc with donor support to strengthen priority setting capabilities, ensure comprehensive delivery on the core package etc). No country administrations are perfect, including in many donor countries, but we need a better framework for how countries transition from receiving support to deliver essential health services to doing so themselves. This is an active process in many countries today (inc where EA supported orgs operate) and has been exacerbated by the recent squeeze on global development financing. 

Lastly, you touch on some practical challenges with funding marginal services. I agree that donors with country presence will be best placed participate in the coordination mechanisms necessary for a marginal aid approach, but EA individuals or orgs could i) choose to support intermediaries. There are none at present but it’s not hard to imagine existing organisations adopting the approach. For example, the Global Financing Facility (GFF) has done much to work with countries and donors on better coordinated and prioritised investments (with some success and some challenges). ii) they could also choose to support catalytic investments to strengthen country capabilities rather than earmarked aid for specific services. 

I would challenge EAs to rethink the rationale for doing global level analysis to set health priorities in low income countries. In absence of local prioritisation it can be better than nothing, and initiatives like the Disease Control Priorities Project have been helpful. But such approaches are both technically and philosophically limited, and inferior to building local capabilities. Before too long, and with the growing decolonisation/localisation movements, I expect this top-down approach will seem increasingly out-dated. 

Thanks again Matt. Yes, negative externalities could be a helpful way to think about at least some of those six challenges. 

To your question, in the short term I wouldn't advise individual small donors to change their behaviour. In absence of a coordinated effort to improve donor harmonisation, I support giving based on cost-effectiveness principles and my intuition is not that this kind of giving is a net harm. Our pitch is perhaps to global health institutions - including EA orgs like GiveWell and Open Phil - that we could do better. We don't yet have the institutions that would allow individuals to support the kind approach we outline (essentially TA + harmonised support to marginal services), but perhaps that's something we need... Of course that's a trickier sell but I'm sure some smart strategic comms folk could help.