Yes Peter, my point was indeed that EA's obsession with measurement is, from what I've seen, making it less Effective, and less able to see how to lever the resources within the movement.

EA stood out as a good idea when I first heard about it, but the few encounters I had - mostly in the San Francisco area when I was based there - showed that it wasn't really that effective as its approach tended to miss those leverage points. 

For example - I notice the EA main website features SendWave in Senegal - I don't know them (though I know their equivalents in several other African countries), my question would be whether EA was Effective enough to see the potential and invest in them, or would be effective enough to replicate the idea in other countries and so on. 

 

Thanks Lucas

I should have numbered the points, would make it easier to reference !

I agree with the critique in that post you linked - RCTs will rarely show you what intervention will be catalytic and change things in the long run, it will systematically under-prioritize high-risk, and ignore approaches that require refining to get to scale. This is why I don't pay much attention to GiveWell, though it might catch the first kind of saving (a cheaper well) but GiveWell's methodology - like those of most other charity evaluators - is designed to ignore most of the other questions that determine effectiveness. For example their RCT won't be back in 10 years to see if the wells are still working.

The rest of that post you refer to is about economic arguments (direct versus indirect poverty elimination) - from what I've seen it is probably only partially correct (and I haven't had time to read the whole article) , GDP is lousy measure of "happiness" and the GDP/capita measure also ignores HOW that wealth is spread, inequality not only means that a rich country can have a lot of very poor (e.g. in the US) but inequality is itself a significant cause of unhappiness. Reasonable people could hold reasoned positions on different sides of that argument so I don't want to dive too deep.

In terms of impact measurement, I argue that it mostly collects meaningless numbers based on experience in the field, the measurement is typically designed to gather the numbers the donors want, not the numbers that actually reflect an increase in effectiveness. More importantly collecting and reporting data is a significant cost, often as much as 5-10% of the total budget, and that means 5-10% of the budget is diverted from creating impact to measuring it. Any effective organization is gathering data, but is gathering data that help it determine whether what its doing is effective, so for example it might just take a small time-bound sample - enough to know it should change something, and then measure something different, this is never enough to provide statistics to "impact measurement" hungry donors. Of course, even after asking many times, including publicly at conferences, I have yet to see Impact Measurement Consultants measure their own Effectiveness i.e. the amount of increased impact they generate compared to the decrease in impact from diversion of resources to its measurement.

In terms of "participatoriness", Karen may be correct in some contexts, about services that people have a "Right" to, but there are many counter examples, and in particular a "right" to a service, doesn't mean you have any likelihood of receiving it, which is why philanthropy or development is needed. For example I did a (unscientific) survey looking at health clinics starting about an hour outside Lusaka. Every clinic had some kind of solar system, more than half were non-operational (which goes back to my earlier point about design), more importantly , the clinics weren't doing anything to fix them, they were waiting for someone else to come along and give them another one. If they'd spent even a small amount of money to pay for that system then they'd have looked for help repairing it, or for example in figuring out that one always-on porch light was using about a third of the entire systems capacity.

In terms of Social Enterprises - my point is not principally about what you call "sustainability", its about leverage, if you are smart enough to fund the creation of a business, that can, for example supply clean water at a price people can afford, then your donation is essentially unbounded because an entity that makes 1c per intervention is infinitely scalable while one that costs 1c per intervention only scales as a function of the donor dollars. Such Social Enterprises need investment to get them to the scale where the economics work - they rarely return enough to satisfy a Venture Capitalist, and without investment they will never get to the point where they look good in impact/spend, but if you measure future impact against the investment to reach scale then I bet many SEs are far more Effective than almost any charity.

On the last point, Its good to see something like Charity Entrepreneurship - but I'd take it one step further, a lot of challenges in global poverty are that a solution exists in the west, but is far too expensive to universally deploy in a poorer country - this is especially true for medical devices. Developing cost effective solutions is very often possible - I've seen many in my work - but I'm asking whether EA would be Effective enough to see that investment (in the R&D to put together the solution) as more effective than continuing to supply the more expensive western solution to a very few recipients.

• Mitra