My experience is that when people use words like effective altruism, innovation or measurable, it is often just buzz words.
Lets assume you want to get water to villages
And lets say you notice that wells cost about $2500.
Is your altruism effective enough to notice who is building them for $2200?
Have you measured the return on impact well enough to know that $100 is the cost of the measurement, that it mostly collects meaningless numbers, and you could dig 5% more wells if you eliminated that?
Have you looked hard enough to notice that $100 is added to the cost per well, for writing different grant proposals to three different organizations, making videos of 3 minutes, 4 minutes and 5 minutes, if you are a funder have you considered a standardized application with other funders ?
Is it effective enough to notice that you are preferring those organizations who already have enough education to prepare a grant proposal; enough savings to fly to the conference where the donors hang out; and who speak good English;
Have you noticed that as many as half the wells, or solar panels or toilets sit there broken waiting for the next donor.
Are you measuring the right thing, would you notice if those $2000 wells fail in 5 years while $2500 wells might fail in 10 ?
Are you innovative enough to figure out that if you got the village to invest $1000, not only could you support twice as many villages but the village would be more likely to maintain it and use it, if they had skin in the game. Are you flexible enough to create a social enterprise rather than a charity, and to fund its overheads rather than expecting it to make a profit?
Are you effective enough to notice that you could be 10x more effective if instead of selling wells to villages, you focused resources of finding and supporting local entrepreneurs to build their own businesses doing so ?
Those wells are still going to fail, and instead of a plaque listing a donor in New York, would it be more effective if there someone local to complain to? So are you effective enough to realize that training local mechanics is part of the investment cost in the local entrepreneur ? Or maybe even training the mechanic to sell the wells ?
Is your thinking long term enough to finance the purchase of a machine that brings the well cost down from $2000 to $1500, even if it costs two or three years to pay back the cost of that machine (and therefore, its purchase couldn’t be funded locally at interest rates of 25-30% per year, even assuming the purchaser had land to put up as collateral)
Are you innovative enough to invest in someone developing a cheaper machine, or one that reduces the cost even further, or are you demanding certainty and measurability too much to consider them anything that pays off at scale, but in the long term.
Are you innovative enough to support the organizations that mentor and support those inventors and engineers, after all with the amount of leverage they generate, it is almost impossible to measure how much they impact the space.
And most importantly, in this context, is EA effective enough to really critique its own effectiveness ?
---
About me: I'm a serial social entrepreneur; My first project was APC.org in the late 80's, now an NGO active in about 74 countries. My last company was Lumeter which was responsible for technology that resulted in lights & cellphone charging (at least) for 80,000 households before it was merged into Mobisol; these days I'm focused on mentoring innovators with scalable solutions for world problems (e.g. the SDGs); for example I'm on the board of foliawater.com, an affordable scalable solution for access to clean water.
I've known about EA for a long time, but I'm not aware of the current state of EA, because as someone interested in scalable "altruistic" solutions it hasn't been "effective" for me to spend time engaging. I came to this conclusion because the limited contacts I had with EA showed a disturbing lack of ability to see the big picture such as described above, and an obsession with "measurement" which pretty much ruled out anything innovative and scalable.
An EA member invited me to post this, which I'd sent them privately, I'm glad to see some introspection at EA so happy to share it in the hope it improves thinking about EA's own effectiveness.
Thank you for writing this Mitra, it's always valuable to hear critiques of current approaches in the EA community. As Peter noted above, your experiences and views would be greatly valued by the community.
I will attempt to respond to some of these questions, but note that my responses may not reflect the views of everyone in the community, and I may miss some crucial points.
Hopefully this didn't come across as dismissive, but I think it's worth giving due credit to GiveWell and other members of the EA community.
All the best,
Lucas
Thanks Lucas
I should have numbered the points, would make it easier to reference !
I agree with the critique in that post you linked - RCTs will rarely show you what intervention will be catalytic and change things in the long run, it will systematically under-prioritize high-risk, and ignore approaches that require refining to get to scale. This is why I don't pay much attention to GiveWell, though it might catch the first kind of saving (a cheaper well) but GiveWell's methodology - like those of most other charity evaluators - is designed to ignore most of the other questions that determine effectiveness. For example their RCT won't be back in 10 years to see if the wells are still working.
The rest of that post you refer to is about economic arguments (direct versus indirect poverty elimination) - from what I've seen it is probably only partially correct (and I haven't had time to read the whole article) , GDP is lousy measure of "happiness" and the GDP/capita measure also ignores HOW that wealth is spread, inequality not only means that a rich country can have a lot of very poor (e.g. in the US) but inequality is itself a significant cause of unhappiness. Reasonable people could hold reasoned positions on different sides of that argument so I don't want to dive too deep.
In terms of impact measurement, I argue that it mostly collects meaningless numbers based on experience in the field, the measurement is typically designed to gather the numbers the donors want, not the numbers that actually reflect an increase in effectiveness. More importantly collecting and reporting data is a significant cost, often as much as 5-10% of the total budget, and that means 5-10% of the budget is diverted from creating impact to measuring it. Any effective organization is gathering data, but is gathering data that help it determine whether what its doing is effective, so for example it might just take a small time-bound sample - enough to know it should change something, and then measure something different, this is never enough to provide statistics to "impact measurement" hungry donors. Of course, even after asking many times, including publicly at conferences, I have yet to see Impact Measurement Consultants measure their own Effectiveness i.e. the amount of increased impact they generate compared to the decrease in impact from diversion of resources to its measurement.
In terms of "participatoriness", Karen may be correct in some contexts, about services that people have a "Right" to, but there are many counter examples, and in particular a "right" to a service, doesn't mean you have any likelihood of receiving it, which is why philanthropy or development is needed. For example I did a (unscientific) survey looking at health clinics starting about an hour outside Lusaka. Every clinic had some kind of solar system, more than half were non-operational (which goes back to my earlier point about design), more importantly , the clinics weren't doing anything to fix them, they were waiting for someone else to come along and give them another one. If they'd spent even a small amount of money to pay for that system then they'd have looked for help repairing it, or for example in figuring out that one always-on porch light was using about a third of the entire systems capacity.
In terms of Social Enterprises - my point is not principally about what you call "sustainability", its about leverage, if you are smart enough to fund the creation of a business, that can, for example supply clean water at a price people can afford, then your donation is essentially unbounded because an entity that makes 1c per intervention is infinitely scalable while one that costs 1c per intervention only scales as a function of the donor dollars. Such Social Enterprises need investment to get them to the scale where the economics work - they rarely return enough to satisfy a Venture Capitalist, and without investment they will never get to the point where they look good in impact/spend, but if you measure future impact against the investment to reach scale then I bet many SEs are far more Effective than almost any charity.
On the last point, Its good to see something like Charity Entrepreneurship - but I'd take it one step further, a lot of challenges in global poverty are that a solution exists in the west, but is far too expensive to universally deploy in a poorer country - this is especially true for medical devices. Developing cost effective solutions is very often possible - I've seen many in my work - but I'm asking whether EA would be Effective enough to see that investment (in the R&D to put together the solution) as more effective than continuing to supply the more expensive western solution to a very few recipients.
Thanks for this, Mitra. I only have time for a quick response, sorry.
Sorry to hear that you have had some bad initial experiences with EAs. It's a pretty big and diverse community, so please don't assume that what a few people think is widely representative of what everyone else does.
There is general agreement that we're trying to answer the question: How can we do the most good? There is very little agreement about how best to do that - some people are more action orientated, while others prefer to be more cautious. Those who act, or write about EA, also disagree a lot. You can check out the forum to see ample evidence. It is one of the things that attracts me to the movement, actually!
Note that we are running a contest for criticism and someone has entered your submission. You have done a lot of impressive stuff, and I am definitely invested in hearing your opinions based on that experience.
If I understand you, you are making the point that that obsession with "measurement" can rule out innovative and scalable solutions to problems by over burdening them. I tend to agree that that can happen, and think that most EAs would agree. However, I think that more than most, EAs tend to value measurement as a way to reduce the risk that our intuitions can lead us astray and to ensure that we are allocating resources efficiently. However, individual EA's perspective on the right amount of measurement effort would vary greatly depending on the specific case.
I recommend that you check the main EA website if you would like to get a better sense of what EA is about - you can also get sent some free books if that's more of a preference. There has been a lot of thought behind some perspectives that may initially seem wrong/confusing (or at least seemed wrong/confusing to me when I first encountered them).
Yes Peter, my point was indeed that EA's obsession with measurement is, from what I've seen, making it less Effective, and less able to see how to lever the resources within the movement.
EA stood out as a good idea when I first heard about it, but the few encounters I had - mostly in the San Francisco area when I was based there - showed that it wasn't really that effective as its approach tended to miss those leverage points.
For example - I notice the EA main website features SendWave in Senegal - I don't know them (though I know their equivalents in several other African countries), my question would be whether EA was Effective enough to see the potential and invest in them, or would be effective enough to replicate the idea in other countries and so on.