Update: I've added extra thoughts down below. I still think the arguments I made here are good, but the case for it might be less strong than I thought while writing it.
Context
Recently, the purchase of Wytham Abbey came in the spotlight: The purchase by Effective Ventures of a (15 million pound?) manor house. On Twitter and on the forum, people have held extensive discussions on whether the expense was justified. I personally was very surprised, which is why I made a forum post requesting an explanation. (Currently, I’m still slightly sceptical but I’m happy I now understand the reasoning behind it a lot better.)
Because of the purchase, the idea that Effective Ventures/CEA should be more transparent about large expenses has been brought up more often. I agree with that idea, and try to defend it here.
I added footnotes/links when I have a source for the (counter) argument. They may have been made regarding a different formulation of my proposal or a slightly different proposal, but are nonetheless worth mentioning. I sometimes rewrote arguments and may have misunderstood their original meaning, so don’t interpret my rewritings as the original authors’ intentions. I just want to credit where I got the ideas from. I have no experience at all with funding projects, so there might be practical things I’ve overlooked.
My proposal
All spending above a certain threshold by EA organisations should be publicly explained
- The threshold should be high enough so it doesn’t take away too much time from grantmakers/employees (ex. $500k, 1 million pounds)
- The more concerned people might be about the expense, or the more influential the expense is, the more time and effort should be put into the explanation
- The public explanation should be some kind of cost-benefit analysis and clearly state the reasons for the purchase including positives, potential negatives and counterfactuals in layman's terms.[1] Precise numbers are admirable but not always necessary. “Worst case” and “best case” scenarios with some kind of probability distribution might be helpful.
- The explanations should be published within a reasonable timeframe (ex. within a month after the purchase/grant is made, every quarter,...). This timeframe should be made clear so that people can expect when to get an explanation of something. The sooner after the purchase, the better.
The EA organisations could be: Effective Ventures and the organisations that fall under it (CEA, 80,000 Hours, Forethought Foundation, EA Funds, Giving What We Can, The Centre for the Governance of AI, Longview Philanthropy, asterisk, non-trivial, BlueDot Impact), Open Philanthropy, GiveWell, Animal Charity Evaluators,... I think the case is the strongest for Effective Ventures and CEA since they represent the EA movement, so they should be held to the highest standards.[2]
I haven’t reviewed every organisation. Some might already do a great job. For example, I get the impression that GiveWell and Open Philanthropy do a better job at explaining grants than EA Funds does (except for EAIF they often only use one sentence, even with million dollar grants).
I’m highly confident some variation of my proposal should be done if the grant is made using individual/small donations, I think it’s reasonable to claim you owe explanations to your donors. The case is less strong when the donor of the grant is just one or two people (like with Wytham Abbey/Open Philanthropy), but I’m still quite confident it’s important in those cases too. You may not owe an explanation to individual/small donors, but rather to the EA community as a whole.
Should this be some kind of official rule every EA organisation must follow? No, but I’d be happy to see some organisations (especially Effective Ventures/CEA) try it out. Different organisations can try different thresholds and use different rules for their public explanations.
Similar proposals have been made before.
Holden Karnofsky (co-CEO of Open Philanthropy) has also written interesting thoughts related to this before. It’s a bit different from what I’m proposing but I’m happy to see some overlap.
Arguments for my proposal
Even though I’ve split up the arguments, I feel like they often overlap quite a bit.
Transparency
I’m for the most part a consequentialist, and I think being more transparent generally leads to better consequences. Perhaps you disagree it generally leads to better consequences. But given moral uncertainty, I would argue transparency is also just a very important virtue to strive for. Neither I, or the EA movement is purely consequentialist. I think transparency fits well within the values of effective altruism as described in points 3 and 4 here.
Transparency isn’t explicitly mentioned. But it’s difficult to do open truthseeking if we lack information/details.[3] In my view, transparency is a form of honesty, or at least closely related. Being transparent is also a way to be a good citizen. It is something I'd expect of any charity/NGO that values effectiveness and empirical backing.[4] And sharing information surrounding decisions is helpful for the wider EA community. This way the expenses themselves can also be talked about and openly criticised by the community. I’m not arguing that expenses should be deliberated by the whole community before being spent. Accountability and transparency does not mean voting by consensus or committee.
This comment by lastmistborn captures my thoughts on this really well:
I feel like the value of this kind of transparency would by far outweigh the costs in time and effort by setting a good example and "practising what you preach" alone. As a major representative of EA and a major funding body, CEA should hold itself to the same standards of cost effectiveness and transparency that we would expect of other NGOs. Additionally, having to publish a writeup like that would incentivize rigorous and clear thinking, and seeing the calculations and reasoning in action would make CEA's general approach clearer. I think it's reasonable to expect a report on spending choices and the reasoning behind them ~every 3-6 months if doing it on an item-by-item basis is too burdensome.
Relatedly, having long write ups can help signal thoughtfulness even if people aren’t actually going to read them.[5]
Effectiveness
If EA organisations publish their explanations (the forum is likely a good place for this, aside from their own website), other EAs are able to give feedback/criticism. The organisations can take these into account for the future, which might improve the effectiveness of their next expenses. This alone could make up for the time spent on writing the explanations. Other EA organisations can also benefit from the explanations/feedback, thus improving their spending as well.
Somewhat related: We should be careful about our effectiveness being reduced because of lavishness. 2 or 3 years ago I would’ve agreed that EA’s generally need to spend a lot more on reducing time and increasing comfort to increase productivity. But now I’m starting to get concerned some are taking that reasoning too far, rationalising bad spending and thus reducing effectiveness.[6]
Optics
With any controversial expense you'll get people who become sceptical of EA. That's unavoidable. But it’s good for anything that might be controversial to have some explanation ready, so that at least our target audience (those interested in having an impact, those with more of a scout mindset) will hopefully read it. This way we don't lose out on potential EA's. It's also bad to wait to give an explanation until there's criticism.[7] I wish I didn't have to make the Wytham Abbey forum post and that I could just link an explanation to critics instead. (Though it now appears that the purchase was made through Open Philanthropy and they're just behind on publishing grants, which makes the situation a lot more understandable).
As a general rule when making purchases, EA orgs shouldn’t put too much weight on optics. Like Owen said, it’s more important to do actual good things than things that look good. But in some cases, optics could seriously affect the consequences in expectation and then the expense should be reconsidered. I think having explanations ready would likely help with optics, if only slightly. (With Wytham Abbey, I’m both worried that it wasn’t a cost-effective purchase and that the optics could result in it being even less effective or potentially harmful, but I'm becoming more optimistic).
A highly relevant and great comment by jacquesthibs.
Responding to arguments against this proposal
“Writing things for the public is a large use of time, since you want to be especially careful and good with language.[8] It could slow down grant-making and other projects by the EA orgs too much.”
Thinking through the costs and benefits must be done anyway with large grants, plus writing it down could help with thinking through the expense. The cost in time of writing it in careful and good language is worth it for the sake of transparency. To reduce time spent writing, you can sometimes just refer to more general posts written in the past (ex. similar to what Open Philanthropy did with their explainer on transformative AI).
“Things that are mostly good for 2nd or 3rd order effects may not have numerical cost-benefit analyses, but rather arguments where someone has judged the balance”[9]
I don’t think the cost-benefit analysis always needs to be completely numerical. But as much detail and clarity as possible would be nice.
“I think the idea that things should be justified by explicit expected value calculations, although it sometimes seems like a core EA idea, is not actually a good one. We usually can't predict and quantify the kind of outcomes we are trying to achieve, and such attempts are more misguiding than useful.”[10]
This is an interesting point. Still, organisations can't be making expenses at random and there must be some logic to it that should be able to be explained clearly.
“Sharing explanations publicly will make outsiders believe they’re stakeholders in the decisions”[11]
Maybe outsiders *should* be more like stakeholders. I’m not sure how far that should go, but I’ll refer back to my arguments on “transparency” and “effectiveness”.
My uncertainties/concerns
- If I’m correct, even with regular funds like the EAIF, some grants are kept anonymous or hidden. There could be good reasons for that (ex. infohazards?). I don't immediately have any thoughts on that.
- There might be an incentive to make grants just short of the threshold. This would be concerning, but I believe it could be solved. Perhaps instead of a threshold it can be something more gradual, for example: Everything above $100k should be published, and the higher the amount the higher the minimum amount of words for the explanation should be. (I'm not saying this example is perfect)
- What constitutes an expense? Does ex. “600k to the community health team” fall under it? Or the individual wages/expenses within the community health team? Where do you draw the line? How do you group expenses together? I don't have a good answer to this, but I feel like there has to be some kind of system that would work well.
- Perhaps the threshold shouldn’t be monetary, but rather based on a degree of potential controversy? This sounds very difficult to put into practice though. Perhaps anything above a monetary threshold has to be explained, plus anything under it that might be influential or controversial.
- “It raises the barrier for people to do projects through EA, since it exposes them to bad faith criticism for every single detail from uninformed (and very often bad faith) individuals”[12]. I don’t have a good response to this, it could be a good counter argument.
Conclusion
I greatly appreciate any feedback and thoughts on my proposal. While some specifics of my proposal might be a bad idea, I’m very confident I’m at least directionally right. I hope to see more transparency regarding big/controversial expenses in the future.
Thanks to Pradyumna and Nick Lowry for the feedback!
Edit: Extra thoughts
I had practically finished this post before Claire Zabel's comment. And since the purchase was made a while back I was of the opinion EVF should've already written something about it. But since it's through Open Phil and it has only finalized this year plus with delay in grant publishing, that changes things. It still comes across as a 'failure' (I don't want to sound this harsh) that there is so much delay, but perhaps there are good reasons for it. Still, I hope this post has some usefulness so I decided to publish it anyway, as I would still like to see more transparency and clearer reasoning. But the case is potentially less strong than I previously thought.
Regarding my opinion on the Abbey: I don't have anything really useful to say that isn't mentioned by others. My goal of my first post was to ask a question and gather information, mostly because I was very surprised. I don't have a strong opinion on the purchase anymore and the ones I have are with high uncertainty.
Also I hope this and my previous post aren't seen as 'hate' towards EVF/CEA/OP. I have great appreciation for these organisations and especially the people working for them. Most of the work I've seen from them is amazing. I want to have a positive impact on the world, so if I can improve their decision making and make them more effective that would be great. That's my only goal.
- ^
https://forum.effectivealtruism.org/posts/xof7iFB3uh8Kc53bG/why-did-cea-buy-wytham-abbey?commentId=YgyoxtjhXMZwfqeAM
- ^
- ^
https://forum.effectivealtruism.org/topics/reasoning-transparency and https://forum.effectivealtruism.org/posts/xof7iFB3uh8Kc53bG/why-did-cea-buy-wytham-abbey?commentId=2BqgGgEf3JG5QnHeW
- ^
https://forum.effectivealtruism.org/posts/xof7iFB3uh8Kc53bG/why-did-cea-buy-wytham-abbey?commentId=rihLCJsSXL6cotCDv
- ^
https://forum.effectivealtruism.org/posts/xof7iFB3uh8Kc53bG/why-did-cea-buy-wytham-abbey?commentId=QshRpAXvMM9rWGb9J
- ^
https://forum.effectivealtruism.org/posts/xof7iFB3uh8Kc53bG/why-did-cea-buy-wytham-abbey?commentId=H77PpBSnmD3C4gwTH
- ^
https://forum.effectivealtruism.org/posts/xof7iFB3uh8Kc53bG/why-did-cea-buy-wytham-abbey?commentId=qFFgMoXQnq2t2G7mJ and https://forum.effectivealtruism.org/posts/xof7iFB3uh8Kc53bG/why-did-cea-buy-wytham-abbey?commentId=6euBFNz5NfNadjgoL and https://forum.effectivealtruism.org/posts/xof7iFB3uh8Kc53bG/why-did-cea-buy-wytham-abbey?commentId=nLMZdk8a57HBHz9Kr and https://forum.effectivealtruism.org/posts/xof7iFB3uh8Kc53bG/why-did-cea-buy-wytham-abbey?commentId=2BqgGgEf3JG5QnHeW
- ^
https://twitter.com/ChanaMessinger/status/1602662148156571649
- ^
https://twitter.com/ChanaMessinger/status/1602662509407879168
- ^
https://forum.effectivealtruism.org/posts/xof7iFB3uh8Kc53bG/why-did-cea-buy-wytham-abbey?commentId=RGJowM5y3HjYGmrb9
- ^
https://twitter.com/ChanaMessinger/status/1602662628656308224
- ^
https://twitter.com/NumeriMagici/status/1602669152623333379
I think most of the organizations you're talking about already explain almost all of their major spending efforts in their annual reports, grants databases, and similar. CEA posts an annual update on the forum, and blogs about its plans. Open Phil posts [per comment below, edit to add: almost] all of its grants with some explanation. And they do have only limited details in many cases - but their more detailed anlyses often involve non-public information, such as consultations with experts who don't want to be publicly named, or analysis of grantees' finances and internal strategy documents.
I do not think that this is accurate, I believe that some of their grants are not posted to their website.
Thanks, you're corect - they say they publish almost every grant, and have detailed explanations of most, but that some have less detail, and some are delayed due to sensitivity or undermining the purpose of the grant. See here, and especially the bolded points in the quote below:
"...we have stopped the practice of writing in detail about every grant that we make. We plan to continue to write in detail about many of them. We will try to focus on those that are especially representative of our thinking and strategy, or otherwise seem like they would be interesting and helpful to discuss.
In most cases, when we do not produce a detailed writeup on a grant, we will still include the grant in our database with basic information about the date, amount and recipient, and a link to the relevant focus area so people can understand what broader goals the grant supports.
In some cases, even basic information about a grant (date, amount, recipient) could constitute sensitive information, for reasons including:
In these cases, we will generally delay publishing basic information about the grant until and unless we have settled on a good strategy for communicating about it. We expect these cases to be rare."
"CEA posts an annual update on the forum."
This seems not to be true? CEA posted annual reviews for 2019 and 2020, but the most recent update post I see is for Q2 2021. Of course that was before the name change, so it was what we would now call an EVF update, but I can't see that EVF has ever made an equivalent post.
There is also nothing (I can find) more up-to-date on CEA's own website or EVF's own website. In fact, the best public source of information about EVF appear to be its statutory filings, which are extremely thorough, but still only run to 30 June 2021 (the filing for 2021/2 won't be due for about 4 months). In a sense, that's the system working as intended, but it does seem that an organisation commited to transparency might make some information available both more prominently and timelier.
FYI we just posted our annual review here.
Is the problem a lack of transparency, or a lack of timely information? They aren't able to reveal it before the decision is made, and there is a very high cost to rapid turnaround.
I had practically finished this post before Claire Zabel's comment. And since the purchase was was made a while back I was of the opinion EVF should've already written something about it. But since it's through Open Phil and only finalized this year plus with delay in grant publishing, that changes things. It still comes across as a 'failure' (I don't want to sound this harsh) that there is so much delay, but perhaps there are good reasons for it. Still, I hope this post has some usefulness so I decided to publish it anyway, as I would still like to see more transparency and clearer reasoning. But the case is potentially less strong as I preciously thought. I'll add this info to the post.
I'd be really interested to see you comment on the Wytham discussion page with your current all-things-considered take.
I don't have anything really useful to say that isn't mentioned by others. My goal of the post was to ask a question and gather information. I don't have a strong opinion on the purchase anymore and the ones I have are with high uncertainty.
Money is generally fungible, so in addition to "grant is made using individual/small donations" being a stronger case for your proposal, I think it is also stronger to the extent that the organization as a whole seeks funding from, and is meaningfully funded by, small/medium donors.
I think the Wytham Abbey situation is a success for transparency. Due to transparency, many people became aware of the purchase, and were able to give public feedback that it seemed like a big waste of money, and it's embarassing to the EA cause. Now, hopefully, in the future EA decisionmakers will be less likely to waste money in this way.
It's too much to expect EA decisionmakers to never make any mistakes ever. The point of transparency is to force decisionmakers to learn from their mistakes, not to avoid ever making any mistakes.
Another consideration is that no-one wants to perform the same cost benefit analysis every year on very similar expenses. Although that seems quite resolvable, by not mandating updates on calculations that have barely changed.
Good post