Primary Author: Sanjay Joshi, with thanks to multiple reviewers in SoGive

This was submitted to the GiveWell Change Our Mind Contest. 
 

Exec summary

1. We believe that GiveWell could improve their modelling of fungibility by (a) adjusting specific details relating to SCI and GiveDirectly; and (b) adding more sophistication to their model.
2. SCI Foundation does a package of interventions, not just Mass Drug Administration (MDA).
3. There is a risk that donations to SCI could “funge”, i.e. the end effect of the donation could be to generate more non-MDA work (even if the donation was restricted to MDA).
4. GiveWell has a row in its model to account for this scenario, however for SCI it is set to zero, meaning GiveWell thinks there is zero chance of this scenario.
5. We believe this is incorrect, although we don’t have access to the data needed to construct a better figure. We think the correct number is in the range 1%-100%, and our placeholder figure is 10%. 
6. Using a 10% within-org fungibility figure leads to a 13% decline in cost effectiveness, from 13.5x cash to 11.7x cash.
7. We also disagreed with GiveWell’s within-org fungibility figure for GiveDirectly, however the extent of our disagreement here was more modest (4% improvement in cost-effectiveness).

(0) We believe that GiveWell could improve their modelling of fungibility by (a) adjusting specific details; and (b) adding more sophistication to their model

We have identified areas of improvement relating to SCI and GiveDirectly. The fact that we haven’t identified areas of improvement for other charities doesn’t necessarily mean that we fully endorse the adjustment size in each case, but may be more a reflection of our time constraints, as set out in Appendix 2.

As well as specific details about SCI and GiveDirectly outlined below, we also argue that GiveWell could be more sophisticated about the size of the adjustment, rather than always using an adjustment of 5%. This would involve considering the specific circumstances of the organisation being modelled.

(1) SCI Foundation does a package of interventions, not just Mass Drug Administration (MDA)

SCI is seeking to grow its work in areas which will enable it to support the WHO goal of elimination of schistosomiasis and other parasitic infections. 

The work of conducting Mass Drug Administrations (MDA) – what many in the EA community think of as deworming – is typically associated with control of worms. 

Alongside continuing to support MDA, SCI is increasingly focusing on a series of cross-cutting approaches to help it move from just controlling NTDs to eliminating them, in line with WHO goals. Such approaches include:

• environmental and behaviour-change interventions to support safe water contact (including WASH-related interventions); 
• morbidity management and prevention for groups not reached by traditional mass drug administration programmes, such as women at risk of female genital schistosomiasis; and
• veterinary public health interventions to address human-animal disease transmission (a One Health perspective).

The cross-cutting interventions intended to achieve elimination are typically not the MDA intervention modelled by GiveWell.

(2 & 3)There is a risk that donations to SCI could “funge”, but GiveWell models this as having zero chance of occurring

GiveWell’s cost-effectiveness model already acknowledges the possibility of “funging”, or what is sometimes also called “illusory restrictedness”. This refers to the fact that a donation to a charity intended for one activity (e.g. MDA to deworm children) might free up funds, with the end result that the charity is enabled to do something else (e.g. WASH work, or another intervention).

This is accounted for in a row in the GiveWell model called “Within-org fungibility”. In the case of SCI Foundation, this is set to zero, and justified in a comment with the words “Not applicable: single program organization.”

As noted above, SCI today does things other than MDA.

Furthermore, as SCI increasingly moves towards the goals set by the WHO, it is possible that interventions other than MDA may grow more quickly than their MDA work. Note that the at-the-margin impact of a donation may at times be more governed by where an organisation is directing more resources to at the margin. This means that even if MDA remains a large portion of what SCI does (and we expect this to be the case), the at-the-margin impact of a donation may be more likely to fund a small but growing part of the organisation's work.

In light of this, we believe that a within-org fungibility adjustment of 0% is almost certainly too low. 

(4) We don’t have the data needed to determine a within-org fungibility adjustment. We think the correct number is in the range 1%-100%, and our placeholder figure is 10% 

As can be seen in Appendix 2, GiveWell typically uses a fungibility adjustment of 5% if it thinks there is any fungibility risk, or 0% for an organisation which does just one thing.

As outlined above, we believe that this is relatively unsophisticated and doesn’t capture organisation-specific circumstances.

We will now expand on how we might incorporate these considerations more carefully for SCI.

The expected extent of funging depends on:

• SCI’s planned spend on MDA
• The extent to which donations are restricted to MDA
• SCI’s planned spend on non-MDA activities
• The extent to which donations are restricted to non-MDA activities
• The proportion of income which is unrestricted

The last of these items (proportion unrestricted) is both relatively easy to monitor and a fairly material driver.

It appears that the proportion of income which is unrestricted has been fairly high in recent years

• c50% in 2020-21 and c75% in 2021-22

It is complex to forecast how this will evolve in the coming years

• In the future, we expect there to still be donations to SCI directly from GiveWell, and some donations from individuals who follow GiveWell’s donations. However we expect the latter category to decline substantially, largely because of GiveWell’s decision to no longer categorise SCI as a Top Charity. The latter is also more likely to be unrestricted.
• Furthermore a material proportion of historic donations have been from GiveWell, and if GiveWell both restricts donations to deworming and encourages other donors to restrict to deworming, this would reduce the proportion of income which is unrestricted.
• To the extent that the unrestricted proportion remains high, this tends towards a high probability of funging.

The amount of donations received by SCI for MDA may decline as a result of them no longer being a GiveWell Top Charity, and organisations typically want to maintain their existing programmes and not see them decline or shut down.

• This may lead to the amount of donations which are restricted to MDA being low compared to SCI’s planned/intended spend on MDA.
• This would lead to a low probability of funging.

We can imagine some scenarios under which the adjustment should be close to 100% (because the money would almost certainly go to non-MDA work) and others in which the adjustment should be close to 1% (because the money would almost certainly go to MDA work).

Given this wide range of uncertainty, it’s hard to select a specific figure that GiveWell should use, and our most confident recommendation is that GiveWell should consider the factors outlined in the appendix and use those to form a view on the within-org fungibility adjustment. 

However if pressed to select a value, we would choose the geometric mean of the upper and lower bounds, which comes to 10%.

It’s worth noting that implicit in this modelling is the assumption that the non-MDA interventions have zero value. This is consistent with the GiveWell approach of having sceptical priors. As part of this review, we have not investigated whether SCI’s work on non-MDA interventions such as WASH may, in fact, be high impact. If it were believed that this were the case, this should be reflected within the model.

(5) Using a 10% within-org fungibility figure leads to a 13% decline in cost effectiveness, from 13.5x cash to 11.7x cash

We determined this estimate of a 13% decline in cost effectiveness by taking a copy of GiveWell’s cost-effectiveness model and changing the “within-org fungibility” figure in the relevant row in the model. The finding is summarised in this part of the spreadsheet, or in the table below:

Table 1: Impact of using a 10% within-org fungibility figure for SCI

(6) We also disagreed with GiveWell’s within-org fungibility figure for GiveDirectly, however the extent of our disagreement here was more modest.

Unlike other organisations where GiveWell has applied a within-org fungibility adjustment, we do not know of any activities which GiveDirectly is conducting other than cash transfers. To justify their decision to include a within-org fungibility adjustment, GiveWell’s only comment appears to be this:

“All funds support cash transfer programs, however some programs may be less cost-effective than the program we model in our cost-effectiveness analysis and there may be some fungibility across programs, particularly in how fundraising staff spend their time.” (Source: this cell in GiveWell’s cost-effectiveness model)

It is unclear to us which programmes they think might be less cost-effective, however we imagine it might be cash transfers to the poor in the US.

However this has generally been in response to disaster events. There has been a coronavirus response fund, however this ended. So it seems questionable that GiveDirectly should have the same within-org fungibility adjustment as other multi-project organisations.

It seems reasonable that the within-org fungibility adjustment should be non-zero, because of the risk that a new project may arise, however this seems to be different from other organisations whether the organisation is known to be doing other projects.

If we change the within-org fungibility figure for GiveDirectly from 5% to 1%, it has the following impact:

Table 2: Impact of using a 1% within-org fungibility figure for GiveDirectly

As can be seen, the impact is not substantial, leading the model to be 4% more favourable to GiveDirectly.

—----------------------------------

Appendix 1: Historic unrestricted income for SCI

We have attempted to determine how much of SCI’s income is unrestricted, and how much is likely to be unrestricted in the future.

As far as we can tell

• 73% of the income was unrestricted in 2021-22 (source: p18 of 2021-22 annual report)
• 48% of the income was unrestricted in 2020-21 (source: p18 of 2020-21 annual report)

This is a fairly substantial change from one year to the next. We have not had time to review this in detail to determine why this is, or what we can expect in future, however it seems that proportions of income which are unrestricted can be ≥ 50%.

Excerpts from the relevant annual reports are below.

Appendix 2: SoGive review of GiveWell within-org fungibility