I agree with this statement entirely.
I was surprised by your “unpleasant to a lot of communities” comment. By that, are you referring to the dynamic where if you have to place value on outcomes, some people/orgs will be disappointed with the value you place on their work?
Not really. I was more referring that any attempt to quantify the likely impact someone will have is (a) inaccurate (b) likely to create some sort of hierarchy and unhealthy community dynamics.
This seems like another area where control groups would be helpful in making the exercise an actual experiment. Seems like a fairly easy place to introduce at least some randomization into
I agree with this, I like the idea of successful groups joining existing mentorship programs such that there is a natural control group of "average of all the other mentors." (There are many ways this experiment would be imperfect, as I'm sure you can imagine) - I think the main implementation challenge here so far has been "getting groups to actually want to do this." We are very careful to preserve the groups' autonomy, I think this acts as a check on our behaviour. If groups engage on programs with us voluntarily, and we don't make that engagement a condition of funding, it demonstrates that our programs are at least delivering value in the eyes of the organizers. If we started trying to claim more autonomy and started designating groups into experiments, we'd lose one of our few feedback measures. On balance I think I would prefer to have the feedback mechanism rather than the experiment. (The previous paragraph does contain some simplifications, it would certainly be possible to find examples of where we haven't optimised purely for group autonomy)
Hey, thanks for this. I work on CEA's groups team. When you say "we don’t know much about which work ... has the most impact on the outcomes we care about" - I think I would rather say
a) We have a reasonable, yet incomplete, view on how many people different groups cause to engage in EA, and some measure on what is the depth of that engagement
b) We are unsure how many of those people would have become engaged in EA anyway
c) We do not have a good mapping from "people engaging with EA" to the things that we actually want in the world
I think we should be sharing more of the data we have on what types of community building have, so far, seemed to generate more engagement. To this end we have a contractor who will be providing a centralized service for some community building tasks, to help spread what is working. I also think groups that seem to be performing well should be running experiments where other groups adopt their model. I have proposed this to several groups, and will continue to do so.
However trying to predict the mapping from engagement to good things happening in the world is (a) sufficiently difficult that I don't think anyone can do it reliably (b) deeply unpleasant to a lot of communities. In trying to measure this we could decrease the amount of good that is happening in the world - and also probably wouldn't succeed in taking the measurement accurately.
Thanks Jona, I agree that there is potential applicability beyond city and national groups - including for individuals.
I'm excited by the results of the A/B/C/D testing!
This is great. It is friendships that kept me anchored in EA when I was in my earning to give phase
After looking into how average temperatures are calculated, I get "Average yearly temperature is calculated by averaging the minimum and maximum daily temperatures in the country, averaged for the years 1961–1990, based on gridded climatologies from the Climatic Research Unit elaborated in 2011."
This means a country like France, that is mostly quite warm, but also has Mont Blanc, where it will be cold at the top, would look significantly cooler than it is.
I think using the average temperature of the capital city is more useful, as this gives you the temperature in the place that the people are (https://en.wikipedia.org/wiki/List_of_cities_by_average_temperature)
Maybe I can give a few thoughts on Chile to help your search.
Not specific to Chile
Thanks Gidon - I really like this post. A few reactions: