Epistemic status: Not a SME and not a data analyst. If mistakes are brought to my attention, I will likely make running changes and pretend it was like that all along.
I'm splitting this into two parts.
Part A is mostly descriptive, in which I share information on next-generation antimalarial long-lasting insecticide treated bed nets (LLINs):
Part B is my attempted analysis of the effectiveness of LLINs with chlorfenapyr:
Traditional LLINs have used a pyrethroid as their sole insecticidal ingredient, against which resistance has become a well known concern. The most promising next-generation LLINs[1] engineered to combat this resistance continue to utilise a pyrethroid, together with one of three additional ingredients:
As of November 2024[3], WHO still recommends pyrethroid-only LLINs generally, and in areas with noticeable pyrethroid resistance[4] they:
In "Part A", I will share my understanding of:
Field RCTs compare performance of next-generation LLINs against pyrethroid-only LLINs. In the table below, I summarise the effects found on two primary outcome measures.
Prevalence is the % of the sample population that has any degree of parasitaemia (detected by rapid diagnostic test and/or microscopy), whether they are symptomatic or not. I believe Prevalence RR (Risk Ratio) is essentially the expected probability of having parasitaemia in that group, divided by the expected probability of having parasitaemia in the reference group (pyrethroid-only LLINs)[5].
Incidence is the number of cases of fever plus parasitaemia per person-year. Incidence RR (Rate Ratio) is the expected malaria cases per person-year in that group, relative to the reference group[6]. It is one minus the reduction in malaria episodes. Going forward, when I describe "effectiveness" it will primarily be by this metric[7].
Age ranges of sampled participants vary, even within the same study for the different outcome measures. However, they are all young children or adolescents, with one exception[8]. The different age ranges are likely relevant to the probability of using LLINs, as well as effectiveness of LLINs given use, but probably not substantially. Relevantly for what follows, both trials involving pyrethroid+chlorfenapyr LLINs enrol 6m - 10 year olds into cohorts for tracking malaria incidence [9].
Distribution performance: Next-generation LLINs versus pyrethroid-only LLINs
Table notes.[10] I have highlighted the large incidence reductions found for pyrethroid+chlorfenapyr LLINs.
All of these trials were cluster RCTs. A significant, and likely majority, of the reduction in malaria illness caused by LLIN use comes from the suppression of infection prevalence in both mosquitos and people[11]. To ensure these community effects are properly captured, and the community effects of surrounding clusters is mitigated, measurements are only taken from participants relatively central in clusters (i.e. inside a buffer zone).
I think this is something potentially crucial to keep in mind. The LLIN products used for each trial were:
The different manufacturers (i.e. behind Permanet, Olyset and Interceptor) use different types of pyrethroids, that have known differences in their lethality and excito-repellency[12]. Furthermore, products from the same manufacturer differ in more ways than the chemicals used. As such, RCT results can be due to characteristics of the LLINs other than their "product class[13]". A few examples:
Testing does indeed show significant differences in longevity between products[21].
While I think this is an important thing to keep in mind when generalising results across LLIN products, GiveWell equate "non-generic" LLINs of the same product class. I do not refute this as the pragmatically best approach - Mosha et al. (2022) do the same in the following CEA - for a rough guess and parsimonious model. However, I ultimately suggest some differentiation in my analysis in Part B.
Mosha et al. (2022)[22] conducted CEA for all LLIN types, using results from the first two years of their Tanzania trial. They ran Monte Carlo simulations using estimated distributions of parameters (both for benefits and costs). All total benefits and costs below are for a hypothetical cohort of 10,000 people covered (1.7 per LLIN).
On the effects side of the equation:
Comparisons of the estimated effects[23]:
These are expected cases, deaths and DALYs for a community of 10,000 people having received each type of LLIN. So lower values are better. YLL = Years of life lost; YLD = Years of life lived with disability [24].
On the costs side of the equation:
Comparisons of the estimated costs[23]:
And combining effects and costs over the two year period [25]:
By advanced analysis techniques[26], I have broken down the ~1750 cases avoided thusly:
As expected, 1 minus the % reduction in cases for children under 10 is practically the same as Incidence RR sizes from the trial for both years.
Regarding the model estimations:
On the model itself:
I am aware of another modelling study into the effectiveness of pyrethroid+chlorfenapyr LLINs that I will be ignoring in this post[32].
GiveWell of course publicise how their model works, if you want it direct from the source I suggest to start by reading this section. The following is a summary, focussed on details relevant for this post.
A meta-analysis found that across 5 field trials from circa 1990s, distributing conventionally treated nets (CTNs) reduced malaria episodes by an average of 45% (compared to control groups not receiving any nets, and with very low existing use[33]). GiveWell take this as the estimated effect on malaria episodes, for under 5s in communities receiving a present day LLIN distribution[34], but with 3 main downward adjustments[35]:
Pyrethroid-only LLINs: The estimates of diminished effectiveness are dictated by averaging results of susceptibility bioassays[38] at a country-level.
Pyrethroid+PBO LLINs: This is largely dictated by the pyrethroid-only LLIN estimates, combined with results from synergist-insecticide bioassays[38]. Basic idea is to estimate what proportion of the accrued pyrethroid resistance is overcome by adding PBO.
Pyrethroid+pyriproxyfen LLINs: Estimates are the pyrethroid-only LLIN estimates due to having "not yet seen compelling evidence"[39] for a significant improvement in effectiveness.
Pyrethroid+chlorfenapyr LLINs: All pyrethroid resistance accrued since the 1990s has been negated.
Bioassays are highly variable, and as such so are effectiveness estimates for LLINs (except those with chlorfenapyr) between countries[40]. GiveWell states that their estimates for +chlorfenapyr (and +pyriproxyfen) LLINs are particularly rough guesses. From my impression, I would describe them as placeholder estimates.
A summary of the GiveWell estimates[41]:
| LLIN type | Scenario | Loss in Effectiveness[42] | Estimated reduction in malaria illness (per CTN-year) |
| Pyrethroid-only & pyrethroid+pyriproxyfen | country average[43] | 42% | 26% |
| worst (Chad[44]) | 62% | 17% | |
| best (Nigeria) | 22% | 35% | |
| Pyrethroid+PBO[45] | country average[43] | 11% | 40% |
| worst (Chad[44]) | 50% | 23% | |
| best (South Sudan) | 4% | 43% | |
| Pyrethroid+chlorfenapyr | universal | 0% | 45% |
I made graphical representations of these estimations for Chad and Nigeria (sans pyrethroid+pyriproxyfen LLINs[46]). I have found it handy to think in terms of LLIN antimalarial features multiplying together to push down malaria illness rates:
Of these next-gen LLINs, those with PBO were the first on the scene in any quantity worthy of being modelled[47]. As such, the GiveWell model is currently structured to estimate the potentiation-of-the-pyrethroid effects of PBO, and not the hit-them-with-something-else effects of chlorfenapyr.
The current GiveWell estimate for the effectiveness of pyrethroid+chlorfenapyr LLINs is unaffected by levels of pyrethroid resistance. This means that the benefits of chlorfenapyr are inversely proportional to the benefits of pyrethroid (i.e. peach + green arrows are the same length in Chad and Nigeria).
I think a malaria reduction effect caused by the addition of chlorfenapyr can be multiplied on top of the estimated effect for pyrethroid-only LLINs, independently of the effectiveness of the pyrethroid-only LLINs. The logic here is that the addition of chlorfenapyr does not reduce the effect of pyrethroid resistance, but is an additional active component on top of the barrier and pyrethroid.
I describe and give estimates specifically regarding Interceptor G2 LLINs against Interceptor G1 LLINs. I comment on generalising to alternative pyrethroid+chlorfenapyr LLINs at the end.
In short, I have estimated that per CTN-year, distributing Interceptor G2 LLINs instead of Interceptor G1 LLINs reduces malaria illness by 50%.
For Chad and Nigeria, this would look like:
A "CTN-year" is rather abstract[37]. In more concrete terms, I think a distribution of Interceptor G2 LLINs, compared to a distribution of Interceptor G1 LLINs (where no other ITNs exist), can expect a reduction in child malaria illness of roughly:
This means I go through a whole modelling process below to essentially end up with the effect sizes in the relevant trials (and for first 2 years, estimates of the Mosha et al. CEA). As such, you might like to stop reading and move on with your life. Below is some details on the adjustments I made, and why.
Accompanying Google Doc (and sheets) I will reference at times.
I made an adorable[48] model to address a few validity concerns regarding the relevant trials, and arrive at an estimate for the effectiveness of Interceptor G2 LLINs. Beware of false precision, although at the end I subjectively adjust/round down my estimate.
Nearly all effects are described in terms of incidence rate ratios (Incidence RRs). Keep in mind that 1 minus this rate ratio is the expected reduction in malaria illness, and the smaller the rate ratio the greater the effect size.
The following is a summary table that I will attempt to explain in stepwise fashion.
Effect Size Estimates (Incidence RRs):
| effect size inputs |
| internal validity adjustments |
| external validity adjustment (estimate that enables the adjustment via GiveWell model) |
| Vs Interceptor G1 | vs no nets | ||||||
| Basis for G1 effectiveness estimates: | Trial Bioassays | Trial Bioassays | |||||
| Questions for Interceptor G2 effectiveness: | TZ | BJ | Avg. | TZ | BJ | Avg. | |
Year 1 | What is the assumed Interceptor G1 effect at 70% use rates? | 0.69 | 0.84 | 0.77 | |||
| What was the effect for cohort children in the trials? | 0.46 | 0.46 | 0.46 | 0.31 | 0.38 | 0.34 | |
| 1. What was the effect for typical children in the trials? | 0.49 | 0.49 | 0.49 | 0.33 | 0.40 | 0.37 | |
| 2. What would have been this effect if clusters were "pure"? | 0.38 | 0.42 | 0.40 | 0.25 | 0.34 | 0.30 | |
| 3. What would have been this effect at 70% use rates? | 0.41 | 0.47 | 0.44 | 0.28 | 0.39 | 0.34 | |
Year 2 | What is the assumed Interceptor G1 effect at 70% use rates? | 0.75 | 0.87 | 0.81 | |||
| What was the effect for cohort children in the trials? | 0.61 | 0.57 | 0.59 | 0.45 | 0.49 | 0.47 | |
| 1. What was the effect for typical children in the trials? | 0.66 | 0.62 | 0.64 | 0.50 | 0.53 | 0.52 | |
| 2. What would have been this effect if clusters were "pure"? | 0.52 | 0.51 | 0.52 | 0.40 | 0.45 | 0.42 | |
| 3. What would have been this effect at 70% use rates? | 0.52 | 0.54 | 0.53 | 0.40 | 0.47 | 0.43 | |
Year 3 | What is the assumed Interceptor G1 effect at 70% use rates? | 0.84 | 0.92 | 0.88 | |||
| What was the effect for cohort children in the trials? | 0.75 | 0.80 | 0.78 | 0.63 | 0.73 | 0.68 | |
| 1. What was the effect for typical children in the trials? | 0.82 | 0.83 | 0.82 | 0.69 | 0.76 | 0.73 | |
| 2. What would have been this effect if clusters were "pure"? | 0.64 | 0.76 | 0.70 | 0.55 | 0.69 | 0.62 | |
| 3. What would have been this effect at 70% use rates? | 0.62 | 0.77 | 0.70 | 0.52 | 0.71 | 0.61 | |
TZ = Tanzania; BJ = Benin
Taking the GiveWell methodology[49], I used susceptibility bioassays conducted as part of the respective trials to estimate the effectiveness of Interceptor G1 LLINs[50].
In summary:
| Interceptor G1 effectiveness | TZ | BJ |
|---|---|---|
| Reduction in effectiveness | 37% | 67% |
| Reduction in malaria illness per CTN-year | 28% | 15% |
| What is the assumed Interceptor G1 effect at 70% use rates? (year 1) | 0.69 | 0.84 |
| What is the assumed Interceptor G1 effect at 70% use rates? (year 2) | 0.75 | 0.87 |
| What is the assumed Interceptor G1 effect at 70% use rates? (year 3) | 0.84 | 0.92 |
LLIN use rates are held constant for all years post-distribution to separate the loss in effectiveness via LLIN deterioration from the loss due to LLIN attrition. It is held constant at 70% specifically as that is the presumed use rates behind these GiveWell estimates.
The estimated pyrethroid resistance in Benin is very high. For reference, the theoretical maximum reduction in effectiveness is 73%[51]. In contrast, the estimated pyrethroid resistance in Tanzania seems typical to slightly low (for the regions where AMF operates in sub-Saharan Africa). The upshot is that, by my estimates, Interceptor G1 LLINs are nearly twice as effective in Tanzania as they are in Benin.
These are the effect sizes directly taken from the Tanzania and Benin trials, with one exception.
Hypothetical Incidence RR in 3rd year of Tanzania trial
For the Tanzania trial, incidence was not an outcome measurement for the third year following the distribution. For the model, I assumed an Incidence RR of 0.75, which I think is a slightly conservative estimate. It is loosely based off the prevalence risk ratio and entomological inoculation rate density ratio[52] in the third year, relative to the third year of the Benin trial and previous years of both trials.
Adjustments to the reported trial effect sizes required estimating how to partition effect sizes between community and personal protection, as a function of LLIN use rates. While this is a very rough guess and the model has known flaws[53], the important outputs (on net) are not significantly affected by what I think are plausible guesses for this partitioning[54]. There is a logical reason for this parameter to shift depending on the Interceptor G1 effectiveness estimation, but I kept it constant to keep the modelling simpler.
All adjustments are centred around estimated use rates of study and non-study LLINs. Generally, I gave the self-reported use rates a downward adjustment of 14%, similar to what has been presumed by GiveWell in their model[55]. More specific details are explained below.
The children being monitored for malaria incidence ("cohort children") are not typical participants. First, self-reported measures indicate they would use study LLINs much more frequently than typical trial participants of a similar age[56]. I suspect to a degree this difference is real. Second, they maintain high use rates in the second year, against the typical trend and what seems to be plausible from results of imbedded net attrition and functional survival studies[57]. These particularly high use rates would overestimate the true effect size, which is what I tried to address through my model. No other adjustments were made for my significant uncertainty regarding use rate estimates, nor likely other unknowns regarding atypical behaviours of cohort children[58].
The frequent use of non-study LLINs in the population would cause the trial effect size to underestimate the true effect size[59], due to the community benefits[60] for the cohort children arising from a blend of LLIN types. I've referred to this as "cluster impurity". I attempt to model this using a basic (and not accurate) assumption that any non-study LLINs used are of equivalent efficacy to Interceptor G1 LLINs.
My best estimate is that the trial effect sizes, averaged together[61], underestimated the true effect sizes by 0.06 points in the first year (0.46 to 0.40), 0.07 in the second year (0.59 to 0.52), and 0.08 in the third year[52] (0.78 to 0.70). As is predictable, these estimates were not materially affected by the Interceptor G1 effectiveness estimates[50].
These trial effect sizes likely overestimate effect sizes for distributions under program contexts. Primarily, this would be because the non-cohort trial participants (let alone the cohort participants) probably had higher LLIN use rates than recipients of LLINs under program contexts. At the very least, LLIN use rates are higher than GiveWell's estimates which I am accepting as accurate[62].
I can input my estimate for the effectiveness of Interceptor G2 LLINs into the GiveWell model and allow it to address the downward adjustment needed for the lower use rates under program contexts[63], as well as all other adjustments. I take my estimated effectiveness of Interceptor G2 at 70% use rates, and essentially reverse the process I took to get the by-year effectiveness of Interceptor G1 LLINs (at 70% use rates) out of the GiveWell model. This arrives at my estimates for Interceptor G2 LLIN per CTN-year effectiveness:
| Interceptor G2 effectiveness | TZ | BJ |
|---|---|---|
| What is the assumed Interceptor G2 effect at 70% use rates? (year 1) | 0.28 | 0.39 |
| What is the assumed Interceptor G2 effect at 70% use rates? (year 2) | 0.40 | 0.47 |
| What is the assumed Interceptor G2 effect at 70% use rates? (year 3) | 0.52 | 0.71 |
| Reduction in malaria illness per CTN-year | 71% | 57% |
| Incidence Rate Ratio (vs Interceptor G1) | 0.40 | 0.51 |
These Incidence RRs average to 0.46, representing a 54% reduction in malaria illness against a distribution of Interceptor G1 LLINs, per CTN-year.
I round this off/down to a 50% reduction in malaria illness, which represents an 8% downward adjustment.
Aside from this final rounding off, I am generally not making downward adjustments for uncertainty[64]. A large reason for this is that it is a recommendation for the GiveWell model, pre-adjustments for uncertainty built into it (most clearly, ~-10% for external and internal validity).
I am ironically very uncertain about this decision. I can see the Bayesian logic behind adjusting downwards. Effect sizes for these Interceptor LLIN arms are remarkably large relative to other modern trials and the performance of CTNs versus no nets in the 1990s. While I do not know what my priors would have been, it would have been much less than a 54% reduction (yet my priors would be given significantly less weighting than the results of these trials). A 1-year cRCT in Côte d’Ivoire that is set to conclude next year may impact on my estimate and certainty.
A few general observations:
I have not attempted thorough cost effectiveness analysis, because there is relevant data I am not privy to.
Currently, it seems as if pyrethroid+chlorfenapyr LLINs cost roughly 30-40% more to acquire than pyrethroid-only LLINs[65]. However, the costs of distribution should be similar, if not identical (as presumed in the Mosha et al. model) and we can expect significant changes in demand and competition, that I suspect will drive the acquisition costs down, never to the level of pyrethroid-only LLINs but potentially to the level of pyrethroid+PBO LLINs.
I have used the GiveWell model to test counterfactual situations, where I set:
For clarity, here I am equating my Interceptor G2 LLIN estimate with all pyrethroid+chlorfenapyr LLINs.
% improvements on distributing pyrethroid-only LLINs
Total deaths averted among people under age 5 (prior to supplementary and funging/leveraging adjustments) - row 77 | ||||||||||
Chad | DRC | Guinea | Nigeria | Nigeria | South Sudan | Togo | Uganda | % improvement | ||
| Chlorf adj | Scenario | GF states | PMI states | |||||||
Current GW estimate | All pyr-only ($6/net) | 28 | 105 | 266 | 157 | 92 | 80 | 31 | 111 | 0.00% |
| All +PBO ($8/net) | 26 | 133 | 245 | 134 | 78 | 94 | 66 | 163 | 7.93% | |
| All +chl ($8/net) | 63 | 157 | 309 | 164 | 95 | 109 | 91 | 199 | 36.44% | |
My estimate | All +chl ($8/net) | 88 | 283 | 605 | 313 | 177 | 187 | 161 | 375 | 151.61% |
Half my estimate | All +chl ($8/net) | 51 | 159 | 361 | 200 | 115 | 113 | 68 | 188 | 44.25% |
Cost-effectiveness in terms of unconditional cash transfers (after all adjustments) - row 203 | ||||||||||
Chad | DRC | Guinea | Nigeria | Nigeria | South Sudan | Togo | Uganda | % improvement average | ||
| Chlorf adj | Scenario | GF states | PMI states | |||||||
Current GW estimate | All pyr-only ($6/net) | 0% | 0% | 0% | 0% | 0% | 0% | 0% | 0% | 0 |
| All +PBO ($8/net) | -3% | 24% | -7% | -13% | -14% | 18% | 47% | 34% | 5.91% | |
| All +chl ($8/net) | 61% | 44% | 14% | 4% | 3% | 33% | 78% | 59% | 28.59% | |
My estimate | All+chl ($8/net) | 106% | 152% | 114% | 84% | 83% | 123% | 169% | 176% | 120.13% |
Half my estimate | All +chl ($8/net) | 39% | 47% | 32% | 22% | 23% | 39% | 49% | 51% | 35.30% |
Disclaimer: Knowledge here is especially piecemeal and superficial. I have given it similarly little thought. I will potentially dig deeper, mostly because I find these trials and ITN quality assurance/longevity interesting.
TLDR: For pyrethroid+chlorfenapyr LLINs that are not Interceptor G2 LLINs, reduce the overall estimated effect on malaria incidence by 20% until there is a published assessment on the quality of field-aged samples.
The Interceptor G2 LLIN was the "first-in-class" for pyrethroid+chlorfenapyr LLINs[13]. Future LLINs of the same class do not need to undergo field trials with epidemiological outcomes, in order to be listed as acceptable by WHO for mass distributions[68]. Instead they need to pass "non-inferiority" experimental hut trials (EHTs)[69]. To demonstrate "non-inferiority" the second-in-class product needs to show with statistical significance that they are better than a certain minimum standard relative (and worse) to the first-in-class product (Odds Ratio of 0.7), primarily for mosquito mortality and secondarily for blood feeding inhibition[70]. Trial arms include new LLINs, and LLINs that are washed 20 times to mimic active ingredient/synergist loss with ageing (and nets in all trial arms are holed to mimic wear-and-tear).
The only pyrethroid+chlorfenapyr LLINs that have passed non-inferiority trials and been accepted by WHO are PermaNet Dual (March, 2003) and Yorkool G5 (September, 2024)[71].
The non-inferiority trials for PermaNet Dual were Ogutu et al. (2024) and Syme et al. (2023). Some high-level observations:
I have not found the two Yorkool G5 non-inferiority trials, but from what I can tell from a skim-read of the supporting documents on the WHO website:
This might point towards a suggestion that Yorkool G5 should be estimated as on par with Interceptor G2, and a downward adjustment needed for PermaNet Dual. But I think it is going too far to make distinctions based off inferences from these trials. The entomological outcome measures are far removed from epidemiological outcomes, and so is the mimicked ageing from real ageing.
I believe it is required that EHTs are conducted once actual aged LLINs are available[73]. Provided this includes trial arms of established LLINs, this would provide important data on actual LLIN survivorship (being retained), wear-and-tear and active ingredient retention. It might be kicking the can down the road a bit, but at the same time it seems unreasonable to generalise results of Interceptor G2 LLINs onto these LLINs that have yet to "pass" in the real world. For any LLIN product, I think pending entomological[74] and quality data on field-aged samples, an effectiveness analysis is likely best off:
At the same time, I acknowledge that this might not be a plausible approach where the goal is to get just rough effectiveness estimates.
Looks like human ingenuity got one over evolution.
This is independent and has not been reviewed for accuracy, so naturally all mistakes are my own. However, I learnt much from attempting related analysis while volunteering with SoGive. I thank the organisation and especially Isobel Phillips and Sanjay Joshi for their mentoring and time, which helped me acquire background knowledge.
Thank you for reading (or at least scrolling) to the end.
As GiveWell identified, WHO will not label an insecticide-treated net (ITN) as an LLIN until they have passed certain examinations. GiveWell and WHO thus label these next-generation nets as ITNs (which encompass conventionally treated nets [CTNs] and LLINs). I however will stick to "LLIN" as it describes their intended functionality, and I think a priori expectations. For the purpose of modelling, GiveWell presumes these next-generation nets last as long as the pyrethroid-only nets of the same manufacturers (that have passed the required testing). My labelling is consistent with the language chosen in much of the research (e.g. Accrombessi et al., 2023).
Bajunirwe & Nantongo, 2024: "Chlorfenapyr acts through disruption of the oxidative phosphorylation in the mitochondria, rather than through the neural receptors, and also carries the advantage of no cross-resistance to PYs [pyrethroids] and other insecticides." I added in "physiological" to distinguish between behavioural resistance mechanisms that are unlikely to be overcome be adding additional active ingredients. Furthermore, even considering just physiological resistance, the claim here of "no cross-resistance" is likely not completely accurate, as cuticular thickening and salivary gland proteins may provide non-specific insecticide resistance (Messenger et al., 2021).
This would include all of Sub-Saharan Africa
Potentially important is that nearly all trials (Uganda the exception) reported effects on prevalence in terms of an “Odds Ratio” (OR), which is the relative odds instead of probability. I have converted these ORs into RRs in a way that is rudimentary and likely not completely accurate (especially where the analysis controlled for effects of baseline prevalence), but I think accurate enough for this purpose as I primarily will focus on Incidence RR. The formula I used was:
Prevalence RR = OR * p(negative | study group) / p(negative | reference group). See Tamhane et al. (2016) p.10
For understanding the difference between Odds, Risk, Rate and Hazard Ratios (see next footnote), I found short explanatory articles by Knol et al. (2011) and Bland & Altman (2000) helpful.
The Benin trial uses a “Hazard Ratio” instead, which I am treating as being equivalent to the Rate Ratio. Unlike rate ratio, the hazard ratio does not require the assumption that the rate is consistent over time, but they can be interpreted the same way (Knol et al., 2011).
And by GiveWell's presumption, this equates to the effect on death (at least when it comes to children under 5). In linked Google docs I might discuss effectiveness on a different outcome (but hopefully this will be clear)
For prevalence, Accrombessi et al (2023) samples from any age.
It is likely that children under 5 are more likely to use LLINs than the population average (e.g. Olapeju et al. 2018, World Malaria Report 2022; c.f. Rek et al. 2020). However older children and young adults are less likely than the population average (e.g. Olapeju et al. 2018, Rek et al. 2020). I treat the effect on 6m - 10 year olds the same as the effect on under 5 year olds (for whom it is likely more effective, but my best guess is that it is in the realm of 1-2 percentage points).
italics = average of 2 point-in-time measurements. * = statistically significant @ p < .05 (when an average of 2 point-in-time measurements, this is when either of the individual measurements were statistically significant; when reported with OR[5], this is regarding the OR). Year given is approximate year of the distribution (sometimes distributions are in batches). Data collated here.
OlySet (and the pyrethroid it contains) has been shown to have significantly higher repellency effects than Permanet (and the pyrethroid it contains), and this might be because they are Type I and Type II pyrethroids respectively (Kawada et al., 2014). If this is true, Interceptor nets likely have low repellency, since alpha-cypermethrin is a Type II pyrethroid (and chlorfenapyr has very low repellency). I think low repellency might be a beneficial trait, but this is speculative on my part as I have no source and in contrast I have seen repellency typically considered to be a good thing.
However, Martin et al (2024) provided empirical evidence that Olyset Plus physically deteriorates much more quickly than what is expected of an LLIN. That said, Lukole et al (2022) shows Olyset Plus to be comparable to pyrethroid-only LLINs, although the paper raised longevity concerns for LLINs generally.
I add “G1” to distinguish them from G2. Elsewhere, you will likely see them called Interceptor LLIN.
https://agriculture.basf.com/global/en/business-areas/public-health/products/interceptor-g2.html. (I have not found how exactly the material differs from G1 to G2.)
E.g. Martin et al (2024)
Mosha et al. (2022). Everything referred to in this section can be found in the paper's Supplementary Appendix (pp. 23 - 30). But output and more information on main Monte Carlo analysis is to be found in the main paper.
Mosha et al. (2022). Supplementary Appendix. S23
I am quite sure both YLD bars are there, and just hard to make out because they are so thin
Mosha et al. (2022). Supplementary Appendix. S24b
I've eyeballed the "cases" bar graph
(4600 - 400) / 2850. Benefits over Olyset Plus (that is the branding of the pyrethroid+PBO LLINs used) are nearly entirely due to outperformance in the second year. Martin et al (2024) is a fabric integrity study nested in this trial, which demonstrated that Olyset Plus deteriorated at a significantly faster rate (and what is expected of an LLIN) than the alternative study LLINs.
(4600 + 100) / 2850
1,759 / (10,000/1.7)
Model entry, and write-up
..aside from this footnote. Many LLIN products are only assessed for performance in "experimental hut trials" (EHTs) on entomological outcomes (e.g. mosquito mortality and blood-feeding success). Performance under realistic field conditions on epidemiological outcomes (e.g. malaria prevalence and incidence) is only needed for the first-in-class products. I give more details behind this under "Generalising to other pyrethroid+chlorfenapyr LLINs".
Sherrard-Smith et al. (2022) created a model to predict epidemiological outcomes in real-world settings given entomological results from EHTs, which Churcher et al. (2024) used to predict the epidemiological outcomes for both the Tanzania and Benin trials given entomological data on pyrethroid+chlorfenapyr LLINs . In short, the model did well predicting the benefit of pyrethroid+chlorfenapyr LLINs over pyrethroid-only LLINs at reducing incidence in the Tanzania trial, but predicted a much greater effect in the Benin trial than was observed (74% reduction versus 36% reduction).
I know this modelling has many limitations and uncertainties, but it also addresses some major limitations of field RCTs. So for CEA, it could plausibly be very valuable (as far as I know, GiveWell only uses similar modelling to validate their pyrethroid resistance modelling). However, I do not understand EHTs nor this modelling enough to say anything more at this stage.
In a hypothetical scenario where alternatively the recipients do not currently have, nor will the receive, any bed nets.
Adjustments I am ignoring are internal validity adjustment (-5%), external validity adjustment (-5%) and "adjustments affecting cost per life saved". To simplify, and because I do not bring them up again in this post.
Cell H22. And Section 3.3 of write-up.
A "CTN-year" can be understood as a year where the average ITN use rate is 70%, and the ITNs are in comparable condition to the meta-analysis CTNs in terms of physical integrity and pyrethroid bioavailability.
GiveWell typically call this a "coverage year" in their model and any explanation. Not sure if I picked up "CTN-year" from any GiveWell document, or I made it up, but I like it.
LLINs give less than 3 CTN-years due higher wear-and-tear and attrition rates. Diminishment in efficacy from insecticidal decay over the 3 years of an LLIN is estimated to average out with the more rapid insecticidal decay, but 6-monthly replenishment, of a CTN.
In short, mosquitos are exposed to twice the dose needed to kill susceptible mosquitos, and mortality rate is counted. My summary of types of bioassays, which do not cover innovations for testing pyriproxyfen and chlorfenapyr.
GiveWell highlights this as an issue in this forum post
Due to GiveWell attributing 27% of benefits to the physical barrier of the nets, the theoretical maximum this can be is 73%.
A simple average across the 7 countries in the overall CEA
Chad is in the GiveWell model due to previously considering funding campaigns, but they decided against it
Effective estimates are for areas where PBO is determined to provide an advantage (which is where there is pyrethroid resistance and the resistance mechanism PBO combats has been detected)
I will not be referring to these much. I do not expect them to play a big role going forward, and I predict WHO to downgrade their recommendation for them due to higher costs without any clear benefit.
I believe comfortably surpassing pyrethroid-only distributions via AMF
Potentially lousy, but it's mine and I gave it some pretty colours.
Although personal judgements were needed to adapt the estimates for this purpose.
I also took a look with Interceptor G1 effectiveness based on GiveWell's estimates for Togo and Nigeria. You likely see them if you take a look through any Google doc/sheet I link. They are largely being ignored within this post.
Due to 27% of the benefits of circa 1990s CTNs presuming to be due to the physical barrier of the net.
Entomological inoculation rate is the rate of infectious bites. See my Google doc for more details.
I use some old modelling of personal vs community effects on entomological inoculation rates (EIR) as a rough proxy, but the relationship between EIR and malaria incidence is not linear. Also, I presume the additional benefit from personal protection is constant at all coverage levels (as a % reduction), but it probably decreases as coverage increases.
Very roughly, I would say that there was 3 percentage point differences between the spans of subjective 80% confidence interval.
For the circa 1990s CTN studies, GiveWell estimated approximately 80-82% average self-reported use rates, and essentially downgraded this to 70%.
Especially Benin, where they reported essentially 100% use rates.
In the Tanzania trial, a nested study by Martin et al. (2024) found 21.1% of Interceptor G2 had been discarded or were too torn for practical use by the end up the 1st year, and this grew to 43.2% by end of the 2nd year . For the Benin trial, a similar study by Ngufor et al. (2024) seems to have found similar results, but I am a little confused over how these figures have been measured and have not invested the time to understand.
Of the adjustments I make, I am least confident about my interpretation here. I contacted the POCs on the papers about this uncertainty but did not hear back.
Churcher et al. (2024) shares this observation with me in their modelling study using entomological data to predict the results of these field RCTs[32]. At first glance, their adjustment is similar but slightly greater than mine.
Not the personal benefits, as the model assumed cohort children were never using non-study LLINs.
Go to the spreadsheet if you want to view the trials individually.
GiveWell has empirical support for their estimate, using actual program distribution data, that I have not looked into. I am accepting their estimations. They are likely accurate and I want to focus on comparison of LLIN types. I think these next-gen RCTs are giving some indication that use rates have increased since the 1990s, but they are also probably elevated (in the Benin trial, hang-up campaigns were run 1 and 7 months post-distribution). There may be other support for an increase in net use rates since the 1990s: Bertozzi-Villa et al. (2021) stated (the trend is likely important, but ignore absolute value of figures due to differences in definitions and measurement): “The use rate has increased with small fluctuations across the time series examined, from a low of 71.3% (67.6–75.1) in 2004 to a 2020 estimate of 87.1% (83.1–90.3).
If you look into my model, you will see I alternatively make this adjustment by directly reading the Incidence RR from the model (using the use rates for the year indicated by GiveWell, rounded to nearest 5%)
Most I do is err towards conservatism on small things, such as treating these effects for children under 10 as being applicable to children under 5[9], and being perhaps naively trusting of the very high study LLIN use rates reported by cohort children.
https://www.theglobalfund.org/media/5861/psm_llinreferenceprices_table_en.pdf.
2.56/1.95 = 131%; 2.65/1.95 = 136%
2.84/2.08 = 137%; 2.94/2.08 = 141%
For CEA reasons only. I am sure that AMF will not be distributing 100% pyrethroid+chlorfenapyr LLINs anytime soon and likely never (researchers often warn about flooding the ecosystem with these LLINs in order to manage chlorfenapyr resistance).
Absolute costs do not matter for the pre-leverage/funging percentage changes for under 5 mortality.
They do for the post-leverage/funging percentage changes for units of unconditional cash transfers. The percentage change is negatively associated with the absolute price. Hence the reason for realistic but slightly conservatively-high estimated costs. I have not attempted to understand leverage and funging calculations.
Known as being "prequalified".
More accurately, they need to pass non-inferiority "semi-field trials", all of which I have seen are EHTs. However, I think I have seen implications of semi-field trials that are not EHTs.
these are entomological outcome measures
I believe this artificial ageing technique has questionable validity to real world, retention does not necessarily equate to bioavailability on the surface, and does not track linearly with effectiveness.
The product’s inclusion within the policy class should be re-examined once experimental hut trial data become available for products aged for 3 years under user conditions (page 7).
I believe you cannot expect field trials with epidemiological data