My reflections on 5 criticisms of FarmKind’s bonus system:

Hello!

After receiving impassioned criticisms on our announcement post last week, I decided to use a plane trip (I’ve been on leave) to reflect on them with a scout mindset to make sure Thom and I aren’t missing anything important that would mean we should change our approach. I’m glad I did, because on my way back from leave I noticed this new post. I thought it would help to share my reflections.

To set expectations: We won’t be able to continue engaging in the discussion on this here. This is not because we consider the “case closed”, but because we are a team of 2 running a brand new organization so we need to prioritise how we use our time. It’s important (and a good use of time) for us to make sure we consider criticisms and that we are confident we are doing the right thing.^[1] But there is a limit to how much time we can dedicate to this particular discussion. Please enjoy the ongoing discussion, and apologies that we can’t prioritise further engagement with it :)

Before I get to my reflections, I want to point out an unhelpful equivocation I’ve seen in the discourse: Some of the comments speak of donation matching as if it’s one very specific thing, and they claim or imply that we are doing that one thing. In reality, “donation matching” can refer to a wide range of arrangements that can work in all sorts of ways.

The most common form of donation matching that I’ve seen involves a large individual or institutional donor promising to match all donations to a specific charity 1:1 until a certain date. This can be fully counterfactual (if the large donor chooses to participate in the match where they wouldn’t otherwise have given to that charity or any other) or it can be not counterfactual at all (if they would have given that amount to that charity no matter what), or somewhere in between. What we’re doing is different to this, as we explain here and below, which is why we don’t call it donation matching, as we aren’t wanting people to import expectations about how it works from previous experiences they may have had with common forms of donation matching.

It’s fair to have qualms with how some forms of donation matching work (or even all forms of it, as some commenters do). But most of the reasons given for concern about donation matching apply to some and not all forms of it. We would be able to make better sense of this collectively by being precise about what forms of donation matching we do and don’t have problems with and for what reasons.

Okay, on to my reflections on 5 criticisms of FarmKind’s bonus system:

Criticism 1: It’s not possible to donate $X in total and have >$X go to your Favorite Charity, which means we’re not offering a ‘meaningful match’ / we’re not delivering what we promise

We received this criticism: “The fact that you can't donate $X and get more than $X going to your favourite charity means I don't really feel like my donation is being meaningfully matched”.

We never promise or imply that it would be possible to donate $X in total and have >$X go to your Favorite Charity (after splitting and receiving a bonus). We make it very clear that it won’t:

Our comms make it very clear that FarmKind’s purpose is to help users donate to fix factory farming through our Super-effective Charities, with the ability to support your Favorite Charity at the same time and to get a bonus included as perks. It’s hard to see how anyone could read our comms and see things the way the critics are worried they will, and so it’s hard to see how anyone could be misled:

Criticism 2: Our comms suggest that your Favorite Charity receives money from the Bonus Fund, but that isn’t true

We received this criticism: "The 'bonus' is presented to users as if it (a) will go in part to their favorite charity and (b) is money that would not otherwise be going to help animals, but neither of these are true"^[2]

We think this is incorrect, because it quite literally IS true that the Favorite Charity receives money from the Bonus Fund. This is how the money flows: Every.org splits the regular donors donation between their chosen Favorite and Super-Effective charity and then Every.org disburses money to each from the Bonus Fund in the exact way that’s summarised during the donation process:

Critics have pointed out that the effect of these cash flows is the same as a different set of cash flows where more of the donor’s money was given to their Favorite Charity than indicated (e.g. $90 of the $150 in the donation went to the Favorite Charity), and all of the bonus went to the Super-Effective Charity ($30 in this case). Those critics claim that this is a simpler set of cash flows, and a simpler/clearer way to understand what has happened. We disagree that it’s simpler or clearer, and even if it were, it wouldn’t be what actually happened, and it wouldn’t make our description misleading.

Criticism 3: Our comms are misleading regarding whether the bonuses we add to donations are counterfactual (criticism fleshed out below)

This is an interesting and nuanced one. The criticism goes something like:

• The average donor assumes that in donation matching, the matching funding added to their donation would not have otherwise…
  • (a) been donated OR
  • (b) been donated to the same cause OR
  • (c) been donated to the same charities
• Because we will not promise a bonus for a donation if the money isn’t already in the Bonus Fund, and if no donations were ever made that money would eventually go to the Impact Fund, our bonus system doesn’t work the way the average donor assumes it does, and we don’t make this clear enough, and so it’s misleading

There are valid aspects of this criticism, which I nonetheless disagree with, and there are invalid aspects. Let’s start with what the criticism gets right and wrong about the reality of the counterfactuality of the bonus system:

Counterfactuality of the bonus system

Regular donations do, in expectation, cause more counterfactual donations to occur. Let’s talk through it, considering counterfactuals in both the narrower/direct and the broader/indirect sense:

1. The narrower/direct sense — i.e. Thinking just about the money that’s already in the bonus fund when a regular donation is made:

• (a) Total donations: At the moment that a regular donation is made, that donation doesn’t increase the total amount of money given to charity, because the funds in the Bonus Fund have already been given, and if no more donations ever occurred and FarmKind shut down, we would give the balance of the Bonus Fund to our Super-effective Charities (as explained at farmkind.giving/support-bonus-fund) ❌ no direct counterfactual impact on total donations ❌ 
• (b) Causes: The regular donor gets to pick any Favorite Charity, from any cause, and their donation will cause money from the Bonus Fund to go to it. Unless by some miracle, the Bonus Fund supporters would otherwise have collectively donated to the same causes as the regular donors in the same proportions, then regular donations do have direct counterfactual impact on how much money goes to different causes ✅ direct counterfactual impact on donations to different causes ✅ 
• (c) Charities: The regular donor gets to pick any Favorite Charity and one of our Super-effective Charities and their donation will cause money from the Bonus Fund to go to it. Unless by some miracle, the Bonus Fund supporters would otherwise have collectively donated to the same charities as the regular donors in the same proportions, then regular donations do have direct counterfactual impact on how much money goes to different charities (both ‘effective’ and ‘ineffective’) ✅ direct counterfactual impact on donations to different charities ✅ 

2. The broader/indirect sense — Regular donations on our platform, which receive a bonus, use up the supply of money that’s in the pool. The more regular donations that occur (i.e. the more demand there is to redeem our bonuses), the more that folks who like how our bonus system is motivation donations will want to keep the system going, by donating to the Bonus Fund. In particular, if the Bonus Fund is running low, we will communicate this to people who like our bonus system, imploring them to donate to keep the system going. As such, each regular donation increases, in expectation, the amount of money contributed to the Bonus Fund.

This is like the how buying one of chickens at the grocery store (which the grocer has already bought and so pre-committed to selling or throwing away) increases the expected amount of chicken they will buy next time. If you think that lower demand for animal products can decrease the amount of animal products produced, you should understand how increased demand for bonus funding can increase the amount of money given to the Bonus Fund.

But what would have happened to the money donated to the Bonus Fund were it not for past demand?

• (a) Total donations: That’s what M&E is for, but in all likelihood, as with all fundraising efforts, some money raised would otherwise have gone to charity and some wouldn’t have ✅ indirect counterfactual impact on total donations ✅
• (b) Causes: For the money that would have otherwise been given to charity, the odds are infinitesimal that it would have gone to the same causes and specific charities that the regular donors chose, in the same proportions ✅ indirect counterfactual impact on donations to different causes ✅
• (c) Charities: See above^ ✅ indirect counterfactual impact on donations to different charities  ✅

Do we communicate about it in a dishonest way?

Certainly not. Our communications are accurate, both in the ‘large print’ and the ‘fine print’.

Do we communicate about it in a misleading way?

It is possible to be misleading without being dishonest, however. This can be done by presenting information in a way that leads someone to draw incorrect conclusions, even though the facts themselves are technically true. Typical ways to do this is by omitting certain information, or framing the facts in a particular way.^[3]

To cut right to the chase: Will some people who don’t read our communications carefully think that they’re having direct counterfactual impact on total donations, when they aren’t [as per (1)(a)]? Probably yes, there is always a risk that some people will get the wrong idea. However, we don’t believe that this constitutes misleading communication. Let me explain why:

1. We take reasonable steps to dispel misconceptions, making what we believe are appropriate and ethical trade-offs to ensure honest communication (although people could reasonably disagree):
   • Measures we take:
     • We don’t call what we do “matching”, in order to reduce the association with the most common form of matching campaigns (and common misconceptions that come with them)
     • In order to donate, users must click a box confirming that they agree with our terms of use and that they “understand that the bonus funds are already pre-paid by bonus fund supporters. Learn how our bonus system works here.”
     • If one clicks any of the various links to learn more about how the process works, they’ll be taken to farmkind.giving/support-bonus-fund, where we explain exactly how the system does and doesn’t allow regular and bonus fund donors to increase their counterfactual impact (I won’t quote the explanation here because it’s too long).
     • In the section about the Bonus Fund on our About Us page, we say: “We help you multiply your impact by boosting your donation with prepaid bonus funds… These funds have been contributed by other donors who want to motivate more people to donate to fix factory farming. When you get you a bonus added to your donation through our platform, you get to direct these funds to the Favorite charity and the recommended Super-effective charity of your choosing.”
   • The need for trade-offs in honest communication:
     • Honest communication involves trade-offs between two goals (among others): Having those who read it form maximally true beliefs — this is the ‘honest’ bit — and making your communication sufficiently simple, concise and interesting that people actually listen to /read it — without this, you haven’t actually done the ‘communication’ bit.
     • Everyone we’ve ever encountered makes these sorts of trade-offs, and considers them to be a part of honest communication. Of course, it’s also possible to make the trade-off wrong and fail to communicate or do so honestly. Here are various some examples, some of which you may think get the balance right and others of which you may not:
       • Every product/service’s terms and conditions and how they surface them to users (I think we’ve all seen examples we think are acceptable and others we think aren’t. Do any readers think that 100% of cases where one clicks to agree with terms and conditions without being forced to read them in detail are misleading?)
       • Teaching Newtonian Physics to early high-school students, even though it’s technically incorrect (I think this is acceptable, as it’s simple enough for students at this level to understand and is a helpful building block towards the correct model that uses Special Relativity)
       • I've put two more in the footnotes for brevity's sake^[4]
   • To sum this point up: People who read the donation form carefully as they fill it out (or who is otherwise curious and so clicks around to learn what this “bonus” thing is all about) will understand it and be making an informed donation decision. People who skim it, may misunderstand what’s happening and so not be fully informed. We’ve taken measures to reduce the likelihood of such misunderstandings, because we think it’s the right thing to do for deontological reasons. It would be possible to take more intense measures (like a series of pop-ups reminding donors several times that bonus funds are pre-committed), but given the trade-offs, we think it wouldn’t be right for consequentialist reasons. Different people will reasonably disagree on what measures we ought to take for deontological reasons, and there is no level that is immune from the criticism that they ought to do more.
2. Even if donors have a misconception about how participating in the bonus system increases their impact, they understand that it increases impact. As discussed, despite our efforts, a skim reading donor may believe that getting a bonus is counterfactually impactful in a way that it isn’t, but there are still 5 other ways (2 direct, 3 indirect) that it IS counterfactually impactful, causing more donations to go to the things they care about. This, after all, is the promise that makes receiving a bonus attractive.

Criticism 4: By encouraging both regular and bonus donors to participate in the bonus system to amplify their impact, we’re causing a double-counting of impact which is suboptimal for collective resource allocation

This is also an interesting one, which we’re grateful was raised, and we will incorporate this into future comms.

Such a double count is likely, and is one of the challenges with counting impact using standard counterfactual reasoning (see discussion here and here). These kinds of double-countings are actually really common. For example, whenever two advocacy groups both play a necessary role in achieving a policy change, they will generally both conclude that the counterfactual impact of their work is the full impact of the policy change. This double count of impact can lead to an inaccurate view of the cost-effectiveness of each group, leading to suboptimal grantmaking decisions.

What are we to do about this? Surely the answer isn’t to not encourage both groups to participate in our bonus system — after all, the participation of each group counterfactually causes higher impact. Each group benefits (in terms of increased impact. We don’t think the answer is to introduce Shapley Values or some other more complicated means of impact attribution, which would compromise our ability to communicate the value to each group. Rather, we should probably talk about the increased impact each group will have in general terms, without using specific numbers unless someone asks for them and/or we heavily caveat them. Having received this criticism, we would not use specific numbers the way we did here again.

It’s worth mentioning that, of course, resource allocation is extremely far from perfectly efficient, so the real cost of these kinds of double counts is probably very low. It’s also worth reiterating that this problem applies to nearly every intervention in EA — it is not a particular problem of donation matching.

Criticism 5: FarmKind uses a form of donation matching, and even if fully understood and consented to by all parties, matching does not belong in EA because.. [various reasons]

It is reasonable to disagree about what is and is not ‘EA’. For example, we have similar disagreements with many EAs’ cause area prioritisation, epistemics, use of funding, communications styles. We also see many projects that don’t meet what we see as minimum standards in terms of the existence of feedback loops, theories of change, or measurement and evaluation. 

We personally wouldn’t choose to claim there is no place for these people or projects in EA. We think that there is value in a more pluralistic and diverse EA community that takes many different approaches, all motivated by the honest desire to do the most good we can. 

I’m sure it’s clear from our choice to launch this platform that we think this is a good thing to be doing, and that it’s our attempt at doing altruism effectively. If some folks who disagree decide that FarmKind does not belong in effective altruism, there is likely nothing we can say that will change their mind. That’s ok. 

I don’t have more to say about this criticism except to reiterate that I respect its intention, I disagree and I thank people for sharing it. We considered Jeff’s critique post of Giving Multiplier prior to deciding to launch FarmKind and we were grateful to come across this perspective before rather than after making the decision — it influenced decisions we made for the better.

Thanks for reading, and happy foruming!

1. ^{^}

   To that end, if any new points are made in response that we haven’t considered, we will consider them and may change our mind, but we may not neccessarily report back about this.

2. ^{^}

   Criticism 2 is about part (a) of this statement. We discuss part (b) under Criticism 3

3. ^{^}

   For example, stating "Our product has been used by over 1 million people!", when 99% of people used it once, were dissatisfied and never used it again. The statement is true (and in that sense, not dishonest), but it omits context so that it might lead someone to believe that 1 million people are active, happy users, which isn't the case.

4. ^{^}

   3rd example: The donation platform Ribon using the half-sentence “Ensure free-range living for farm animals” to describe the impact of donating to a specific charity, even though no given donation can possibly guarantee this outcome is achieved (I think this is acceptable, and I’d hope that further comms were available for those who are interested enough to read more to understand how their donations might have an impact).

   4th example: Saying “I like ice cream”, even if you think that the self is an illusion (an illusion which your statement reinforces), such that you’re giving the false impression that you think the self is a real thing (I include this example because I imagine that even our most steadfast critics would agree this is honest communication. The kind of person who caveats their use of the word ‘I’ is probably failing to communicate)

No I don’t think “part of the pot goes to your account” is a fair way to characterize your offer. I think you may have edited your comment as I was responding to it.

It’s clear we agree on what would have otherwise happened to the money already in the bonus fund at the time of a regular donation. I don’t think we need a hypothetical example to dig into that any further. It’s also clear what we disagree on (please see other comments so I don’t have to state it again). I don’t aim to change your mind on those points, and so I’ll leave things there :)

I agree with your interpretation of this case (except for what the most straightforward way to describe it is), but you seem to be missing the broader point about the interplay between supply and demand for matching funding which means that both groups play a causal role in increasing donations to the favorite and super-effective charities alike. I understand you think the way we’ve communicated this is misleading, not in the spirit of EA or otherwise wrong. This is a valuable perspective, for which I thank you, but we respectfully disagree.

I hope what we do now agree on is that, regarding your comment “the site gives the impression that part of the bonus goes to the favorite charity, but that isn't usefully true”, it IS true that part of the bonus goes to the favorite charity.

That’s all the time I have to spend on this topic. I hope to have clarified some of the facts about how the platform works.

That people are willing to donate through a platform that clearly sign-posts that you don’t get >$X donated to your favorite charity if you donate $X split between two charities IS evidence that many donors don’t share Ben’s expectation about how our bonus system should or does work. That’s all I suggested it was evidence for. I’ve only weighed in here to correct false or misleading claims made about how the platform works.

As for whether how the platform works is aligned with EA or not, or misleading or not, I haven’t weighed in because I’m sure it’s clear from our choice to launch this platform that we think this is a good thing to be doing, and that it’s our attempt at doing altruism effectively. It’s reasonable to disagree and I understand and respect your disagreement. I have similar disagreements with many EAs’ approaches, decisions, cause area prioritisation, communications styles and so on. Once there aren’t any remaining misunderstandings about how our platform works, all that’s left is one of those kinds of disagreements. I don’t have much to say about your position except that I respect it, I disagree and I thank you for sharing it. I considered your critique of Giving Multiplier prior to deciding to launch our platform and was grateful to come across this perspective before rather than after making the decision. So thank you again!

If that’s what it takes for you to feel your donation is being meaningfully matched then it’s another reason this platform isn’t the right choice for you. But that’s a very specific requirement that our platform never claims to meet, and that plenty of folks don’t share with you, as evidenced by existing donations through our platform. Many other donors are excited by the prospect of having both their donation to the favorite and super-effective charity receive a bonus. This is what happens on our platform, and is what’s laid out as they step through the process. No one is donating under any misapprehension that after splitting their donation and receiving a bonus they get more than the total amount they donated given to their favorite charity.

We are not “baiting” people in with the promise to cause more dollars than they donate to go to their favorite charity and then “switching” to a split and boost mechanism. Rather, the platform (as we lay out on our landing page, and repeat throughout) promises to allow you to help fix factory farming while supporting your favorite charity too, and to get your donations to both charities boosted. The platform isn’t about getting more than the total amount you donated to go to your favorite charity (it’s about splitting and boosting your donation) so the fact that it doesn’t do that is a feature and not a bug

On your first point, your way of describing things may or may not be clearer, but it’s not accurate. See my response to Ben Milwood about how the money flows. You may see the two scenarios as being the same in effect, but in that case our comms are equally accurate in describing the effect and more accurate in describing the money flows. We’ve chosen what we believe to be the most clear, intuitive and compelling way to explain things.

On the first part of your second point, yes - some bonus funding money would have otherwise been given to help animals (like anyone convinced by this post), while other bonus funding money (and so far the majority) wouldn’t have. So it is not the case that all bonus funds would otherwise go to help animals, which your original critique suggested was the case. That’s what I wanted to correct the claim.

Regarding your more limited claim, I don’t agree that “every dollar donated to the bonus pool is one that will be paid out to effective animal charities regardless of how other donors behave” because dollars in the fund very literally get paid out to favorite charities that needn’t be super-effective or support animals.

Plus, when thinking about the counterfactual impact of the bonus system, I think what’s relevant is what would have happened to those dollars if the fund didn’t exist - not just what would have happened to them once already in the bonus fund. As explained, many donations to the bonus fund would not have been donated at all were it not for the fund or would not have gone to effective animal charities. And were it not for regulator donors generating demand for bonus funding, we could not fundraise for the bonus fund. The two groups of donors incentivise one another to donate and lead to more total donations to effective farmed animal charities than if the system did not exist. We explain this in our comms. I believe that how our system works is in keeping with the spirit of even the most naive interpretation, in that, whether you’re a regular or bonus donor, your participation in the system leads to more money being donated than would have occurred otherwise.

It’s fair enough that you would have chosen to communicate things differently in our position. Thank you for giving the system some thought and voicing your perspective on how you would choose to communicate things! It’s helpful for us to consider such perspectives as we continuously improve our platform.

What’s actually happening in the back end is what is laid out in the screenshot: The donor’s donation is split according to their chosen percentage, and then the bonus fund (which is an independent pool of money held by every.org) is disbursed to the favorite charity and the super-effective one.

Hey Jeff, I just want to correct a couple of false claims: (a) The bonus WILL in part go their favorite charity, as you can see when you step through the donation process (see this screenshot: https://drive.google.com/file/d/175IonBvB7uRDuZUCybJ50njnTMh5NV5n/view?usp=drivesdk). This is true of Giving Multiplier as well. (b) It is not the case that all bonus funds would otherwise go to help animals. We fundraise for our bonus fund from the same donor pool as our regular donors, encouraging them later to “pay forward” the match they received by supporting the bonus fund and encouraging others to donate. For this reason, much of the money in the fund would not have otherwise gone to animals or effective charities. In our case, our bonus fund was seeded by a friend’s birthday fundraiser, where most of the money was donated by their extended family who are not animal people or EAs.

I disagree with your other critiques, but don’t see a productive outcome or use of time in digging into it here, so I just wanted to correct those 2 factual misapprehensions.

Hi David, it’s great to hear you’re interested in supporting the Bonus Fund! I’d recommend playing around with the giving widget on the website to understand what donors get to decide, but in short:

They choose both a Favorite and Super-Effective charity to split their donation between. They can choose any U.S registered charity as their Favorite Charity, and they can choose from 6 individual charities (The Humane League, Fish Welfare Initiative, Shrimp Welfare Project, Sinergia Animal, Dansk Vegetarisk Forening, Good Food Institute) or our Impact Fund (a combo of all 6) or Climate Fund (a combo of the last 3 I listed) as their Super-Effective Charity. The more of their donating they allocate to the Super-Effective Charity (as opposed to their Favorite) the larger the bonus they get, incentivising people to give more effectively and causing the vast majority of matching funding to go to the Super-Effective Charities.

You can also click on “More detail on how it works” at https://farmkind.giving/support-bonus-fund for more detail on how the system adds value.

Fixed - thanks