Cost-effectiveness of iron fortification in India is lower than GiveWell's estimates

Summary

GiveWell considers iron fortification programs in India as cost-effective and has made incubation grants to Fortify Health that runs wheat fortification initiatives in 2018, 2019 and 2021. The effectiveness of iron fortification was assessed to primarily arise from three categories of benefits - 1. averting anemia morbidity, 2. cognitive benefits for children, and 3. cognitive benefits for adults. Two key variables underlying the magnitude of these benefits are — the prevalence of anemia across age and sex groups in India and the proportion of anemia cases that can be attributed to iron-deficiency. 

Recent work in the public health of anemia and data from the Comprehensive National Nutrition Survey suggests a significant update in the above two variables that dramatically decreases the cost-effectiveness of iron fortification programs in India. 

First, WHO guidelines for hemoglobin cutoffs to diagnose anemia have come under scrutiny for not being representative of non White populations. Recent evidence from work seeking to establish Indian population representative hemoglobin cutoffs suggests that the actual prevalence of anemia in different groups in India may be 1/2 - 1/3 of the estimates used by GiveWell's model. This alone reduces the direct health benefits of the $4.5M 2019 grant to Fortify Health by 58%, or 1.2x the effectiveness of GiveDirectly.

Second, work on the The Biomarkers Reflecting Inflammation and Nutritional Determinants of Anemia (BRINDA) project suggests that in countries with a high degree of infection, iron-deficiency may contribute to a smaller proportion of anemia cases. Data from the CNNS provided evidence that this is the case with Indian children/adolescents and it seems intuitive that it also applies to adults.

 Finally, GiveWell assessed that there was no significant risk of harm associated with iron fortification. Emerging evidence suggests a potential for risks that are dramatically higher than assumed, particularly that increases in iron can lead to a rise in non-communicable diseases like type-2 diabetes. Such risks are not covered in GiveWell's report and deserve careful consideration given their magnitude of harm.

Major Sources of Uncertainty

• The role of iron in health and disease, the aetiology of anemia, and fortification as an intervention are vast and nuanced topics with hundreds of papers and meta-reviews that I could not possibly review. I am also not a medical or public health scientist and although I consulted doctors while researching/writing and received positive feedback from a public health scientist on a version of this work, it is possible I'm making some basic mistakes in my interpretation of the literature.
• One of the major empirical leaps I make is that the proportion of anemic adults using new hemoglobin cutoffs can be roughly inferred from the ratio of prevalences b/w 15-19yr olds and adult age groups from the old cutoffs and data of 15-19yr olds from the new cutoffs. That is, I'm assuming the distribution of anemia across age-groups is similar using either old or new cutoffs. This seems like a fair assumption to make for a ballpark estimate but maybe the distributions are weird such that this was unfounded.  This is especially important as GW's model places a very high weight on adult benefits.
• The section on risks from excess iron relies on reading only a few recent papers and the Indian specific evidence comes from a pre-print.  

Background

Anemia is a condition with a multifactorial etiology characterized by a decrease in red blood cells and reliably marked by a decrease in hemoglobin levels. The loss of red blood cells leads to a deprivation of oxygen with numerous physiological, cognitive and economic implications. Anemia during childhood and adolescence can lead to severe developmental issues, some of which may be irreversible. (WHO) 

Anemia is often associated with a deficiency of iron. However, deficiencies in other micronutrients such as the vitamins folate, B12 and A, infections by malarial parasites, worms, and others, inflammation from sub-acute infections, microbiota disturbances as well as hemoglobinopathies (structural defects in RBC’s) are other important factors associated with anemia, many of which have a complex relationship with iron levels. 

GiveWell's report on iron fortification concludes that “There is strong evidence that iron fortification reduces cases of iron deficiency and anemia” and iron fortification is a program they are ‘actively supporting'. GiveWell has made incubation grants towards Fortify Health, with the latest being $8M in 2021 to expand wheat fortification in India. The report notes that the primary driver of the cost-effectiveness (3/4th of the value) comes the from the value of information the grant would provide towards reallocation of future additional funding to FH over less-effective programs.

Current cost-effectiveness modeL

In GiveWell’s cost-effectiveness model for iron fortification, the benefits are broken down into three categories -

1. Benefits from averting anemia-related morbidity (39% of the value) 
2. Cognitive benefits for children (8% of the value) 
3. Cognitive benefits for adults (53% of the value)

In their model for the 2019 grant to FH for wheat iron fortification in India, these benefits were calculated using data from the Global Burden of Disease (GBD) database. The model then factors in the specific demographic of FH's target population, specifically looking at only the states of Maharashtra and West Bengal, and only the top three wealth quintiles. All benefits end up being downstream from estimates of the anemia prevalence in India. Specifically -

• For 1, the model bases anemia-related morbidity on the 2017 estimate of 859 YLDs/100,000, based on 2017 anemia prevalence of 28.4% across ages and sex.
• For 2, the model is based on the estimate that 21.1% of children 0-14 have dietary iron deficiency.
• For 3, the model is based on the estimate that 24.5% of working-age adults have dietary iron deficiency.

GBD anemia data for India was based on 42 sources, with the most recent being the Demographic Health Survey (DHS) conducted in 2016. This survey, as well as the most recent one from 2021, uses hemoglobin level (g/dL in blood) cutoffs based on current WHO standards. 

Updates to cost-effectiveness

Anemia prevalence in India may be significantly lower than reported

Crucially, the WHO hemoglobin cutoffs are under scrutiny for being non-representative of non White populations.  The current cutoffs are largely intact from 1968, where they were proposed based on five studies from North America and European populations. (paper) In a review of 60 global studies between 1975 and 2018, the authors evaluated haemoglobin variation across the lifecycle and found that haemoglobin cutoffs in children and adolescents from African and Asian regions were substantially lower (1–2 g/dL) in some datasets. This prompted the WHO to hold a technical meeting in 2019 where emphasis was given to the need to change diagnostic criteria, with one proposal being "anemia could be defined by hemoglobin levels below the reference range, that is, below a statistical centile (e.g., the 2.5th centile of hemoglobin in a healthy population)."

Following up on this work, a 2021 Lancet paper by Sachdev and colleagues examined data from the 2019 Comprehensive National Nutrition Survey [CNNS], a large-scale, nationally representative survey of children and adolescents aged 0–19 years in India and “constructed age-specific and sex-specific haemoglobin percentiles from values reported for a defined healthy population in the CNNS.” They found that “study cutoffs for haemoglobin were lower at all ages, usually by 1–2 g/dL” that resulted in a large drop in anemia prevalences as compared to WHO findings. 

Overall, the new representative cutoffs lead to a 20 point decrease in anemia prevalence, or 1/3rd the prevalence estimated using WHO cutoffs, for 0-19 yr olds in India. The same was true when excluding 15-19yr olds.

It's important to note that the anemia prevalences using the standard WHO cutoffs differs significantly between the CNNS and the DHS (used by GBD and GW) for the same age-groups. The likely reason is that the CNNS performed a more rigorous assessment using multiple biomarkers and a whole blood count, while the DHS only used capillary blood from a finger prick (known to provide a less robust measure of hemoglobin). 

I performed a very crude calculation to estimate anemia prevalences using the age-group proportions in the DHS data and the 15-19 year old data from both surveys as a bridge, giving the following estimates - 

Roughly, the decrease in anemia prevalence for both males and females across age groups is 1/3rd the original prevalence when WHO cutoffs were used.

To sum up, the basic assumptions of the model would change in the following ways (I'm not accounting for state-wise differences though I expect the effect size to not matter in this ballpark) - 

• The prevalence of anemia in 0-14yr olds is 10.78%, half the previous estimate 
• The prevalence of anemia in adults is 8%, a third of the previous estimate 
• The YLDs from anemia is halved (conservative simplification)

Keeping all the other parameters the same, this reduces the direct health benefits of FH's iron fortification by 58%, leaving the program's direct benefits to be only 1.2x as effective as GiveDirectly.

Proportion of anemia cases due to iron-deficiency may be lower than estimated

Not only are actual anemia prevalences likely lower than estimates used in GW's model, the proportion of anemia cases in India attributable to a deficiency of iron may also be less than GBD data suggests. Recent work, like The Biomarkers Reflecting Inflammation and Nutritional Determinants of Anemia (BRINDA) project, points towards a smaller contribution of iron-deficiency (ID) to anemia in countries with high infection rates.  The authors note that -

our results suggest that, in settings with low and moderate burdens of infectious disease, nutritional factors such as ID are important contributors to anemia. In contrast, in settings with a high burden of infectious disease, the contributors to anemia are more diverse, and both nutrition and infectious disease control are important factors to address.

The authors suggest that anemia mitigation interventions should be tailored towards specific countries and follow detailed risk assessment studies, as well as factoring in inflammation-adjusted iron status indicators.

The 2018 CNNS data once again comes in handy as it includes relevant information needed to identify the prevalence of ID-associated anemia. A 2021 Journal of Nutrition paper by Sachdev et al performs an analysis of the CNNS data for 0-19 yr olds and finds that -

This is the first study from India providing estimates of ID prevalence in a representative sample of children and adolescents at the national and state levels using multiple inflammation-adjusted ID indicators. In preschool children and adolescent girls, ID based on SF adjusted for inflammation by the modified BRINDA method was a public health problem of “moderate” proportions (∼30%–32%), whereas in 5- to 9-y-old children (15%) and adolescent boys (11%) it was a public health problem categorized as “mild” 

The authors claim that these ID-prevalences are lower than other smaller non-nationally representative samples (~63% for 10yr olds, ~50% for 12-16 yr olds).

I'd stress that the above findings pack a lot of nuance. For one, a surprising finding was that ID inferred from serum-ferritin levels was higher in richer and urban states vs poorer and rural states - which on the surface level implies a more optimistic modelling outcome for GW's FH grant (which assumed fortification would reach richer quintiles who would have lower ID rates and thus have a lower impact). However, the authors note one possible reason for this could be a "functional iron deficiency" where

possible inefficient utilization of stored iron for hemoglobin synthesis in children from lower wealth quintiles suggests that iron supplementation alone may be of limited value, and the focus should be on improved diet diversity, which facilitates iron absorption while providing all other hematopoietic nutrients.

This consideration leads the authors to conclude that

Our findings have important implications for iron supplementation and fortification programs. An important conclusion is that given the impaired iron utilization for hemoglobin synthesis in low-SES children, intended benefits on anemia will not accrue if iron intake is increased without also addressing the multiple environmental constraints related to poverty.

I'm unsure how to assess the impact of this section on GiveWell's cost-effectiveness model as I wasn't able to pin down the estimate for iron-deficiency's contribution to anemia in the model. Since the BRINDA method and other work were published from 2018 onwards, and the GBD's data sources are much older, about a decade prior on average, it seems likely that GBD overestimates ID's contribution to anemia.  Like before, the CNNS data is restricted to age groups b/w 0-19 yrs, but intuitively should extend to adult age groups as well.

Risks from iron overload may be higher than expected

In their report on iron fortification, GiveWell notes the potential adverse effect of iron fortification and concludes that not enough evidence exists to suggest that risk is high. In particular, the risks examined were an increased risk of malaria, an increased risk of non-malarial infections, iron overload, and gastrointestinal side effects. On the subject of iron overload specifically, they note -

We have not come across any evidence on the prevalence and effects of iron overload in our reviews of the literature and are uncertain about how common this is. Our guess is that cases of severe iron overload are rare in iron fortification programs because the effective dose is far lower than in most supplementation programs. However, we would consider investigating iron overload in more depth if we recommended additional funding to iron fortification programs.

In their $8 million 2021 report to Fortify Health, they note that they have not “incorporated any additional evidence on fortification published since that [iron fortification] report.” Adverse effects are also not included in their cost-effectiveness model for the 2019 grant.

While I did not extensively examine the literature, a few papers stood out that provided evidence for additional iron, in the range of the effective dose from fortification, had potential to lead to harmful outcomes. Worryingly, these outcomes were not mentioned, let alone assessed, in GiveWell's report.

Specifically, two risks that appear to be associated with excess iron are non-alcoholic fatty liver disease and type-2 diabetes —

Additional iron may increase non-alcoholic fatty liver disease in obesity through the gut microbiome (paper)

Serum ferritin levels, as a markers of liver iron stores, were positively associated with liver fat accumulation in parallel with lower gut microbial gene richness, composition and functionality.

Iron metabolism and type 2 diabetes mellitus: A meta-analysis and systematic review (paper)

A total of 12 case–control and cohort studies were analyzed. Of the 12 studies, 11 described the correlation between serum ferritin levels and type 2 diabetes. The median and high serum ferritin concentrations were significantly associated with the risks of type 2 diabetes (odds ratio [OR] 1.20, 95% confidence interval [CI] 1.08–1.33 and OR 1.43, 95% CI 1.29–1.59, respectively).

Again, data from the CNNS analysed by Sachdev and co provides information for the Indian context. They investigated whether serum ferritin levels (a functional form of iron) was associated with high fasting blood sugar levels (FBS - marker of diabetes), high blood pressure (HTN - marker of hypertension) and high total cholesterol levels (TC - marker of dyslipidaemia). They found an “odds ratio (OR) of high FBS, HTN and TC were 1.05 (95% CI 1.01-1.08), 1.02 (95% CI: 1.001-1.03) and 1.04 (95% CI: 1.01-1.06) respectively for every 10μg/L increase in SF. The odds for high TC increased with co-existing prediabetes. The scenario analysis showed that providing 10 mg of iron/day by fortification could increase the prevalence of high FBS by 2%-14% across states of India. Similar increments in HTN and TC can also be expected.”

The increase in risk of diabetes is especially relevant in India where a large proportion of children (50%) appear to have at least one biomarker of ‘metabolic obesity’ that predisposes them to diabetes.

I didn't try to quantify the magnitude of risk and want to stress that my research and the findings are preliminary/limited. However, the directionality and potential effect size warrants a closer look at the evidence of harm from iron fortification, especially in light of the smaller size of the benefits.

Meta-level, Optics, and Ethics considerations

Three parting remarks  -

• In GW's report on iron fortification, they note that "In our assessment of the effect of iron fortification on iron deficiency and anemia, we rely heavily on Gera, Sachdev, and Boy 2012, a meta-analysis of randomized and quasi-randomized trials of the effect of iron fortification on measures related to iron deficiency and anemia".  The corresponding author of the meta-analysis and the analyses of the CNNS data, including the iron-diabetes linkage, is HPS Sachdev. Mentioning this because it caused me to update positively towards the credibility/quality of the evidence I present here.
• Multiple pieces critical of iron fortification have been published recently in leading Indian newspapers (The Hindu, Indian Express, Times of India). International philanthropy/charities already tend to be viewed with a degree of hostility in India and it's possible EA gets dragged under the bus because of its support for iron fortification.
• I'm worried about the ethics of fortification with respect to the way it takes away agency from the dietary choices of people. This seems okay if we can be extremely confident that the choices being pushed have minimal risks of harm (a priori seems true for chlorination of water and vitamin A supplementation) but iron doesn't seem to as clear-cut. It also seems important that GiveWell consults with medical and public health researchers from respective countries where interventions are being assessed to catch country-specific considerations.

References

GiveWell Iron Fortification report - https://www.givewell.org/international/technical/programs/iron-fortification#Effect_on_iron_deficiency_and_anemia 

Fortify Health 2021 Grant report - https://www.givewell.org/research/incubation-grants/Fortify-Health-expansion-December-2021#Cost-effectiveness 

GiveWell 2019 model

https://docs.google.com/spreadsheets/d/1QZNP8J_TSCjLN1iIuOdupR1l_v4r-789jXm2kBFfyNg/edit#gid=1417746729 

WHO Anemia - https://www.who.int/health-topics/anaemia#tab=tab_1 

DHS 2021 data - 

https://dhsprogram.com/pubs/pdf/FR375/FR375.pdf 

1968 WHO hemoglobin cutoffs - https://doi.org/10.1016/S2352-3026(18)30004-8

Cutoffs varying across population and lifecycle - https://nyaspubs.onlinelibrary.wiley.com/doi/full/10.1111/nyas.14096 

Report on 2019 WHO technical meeting - https://nyaspubs.onlinelibrary.wiley.com/doi/full/10.1111/nyas.14090 

Haemoglobin thresholds to define anaemia in a national sample of healthy children and adolescents aged 1–19 years in India: a population-based study, The Lancet, 2021 - https://www.sciencedirect.com/science/article/pii/S2214109X21000772

Prevalence of Iron Deficiency and its Sociodemographic Patterning in Indian Children and Adolescents: Findings from the Comprehensive National Nutrition Survey 2016–18, The Journal of Nutrition, 2021 - https://academic.oup.com/jn/article/151/8/2422/6287924

BRINDA project - 

https://academic.oup.com/ajcn/article/106/suppl_1/402S/4668592 

Is iron status associated with markers of noncommunicable disease in Indian children?- https://assets.researchsquare.com/files/rs-1136688/v1_covered.pdf?c=1638568122 

Metabolic obesity in Indian children - https://pubmed.ncbi.nlm.nih.gov/33893450/ 

The Hindu Press piece - 'Inescapable risks of mandatory iron fortification' by Kurpad and Sachdev - https://www.thehindu.com/sci-tech/science/ines

Times of India - Report says iron-fortified rice unscientific and risky https://timesofindia.indiatimes.com/city/raipur/report-says-iron-fortified-rice-unscientific-and-risky/articleshow/92349468.cms

Indian Express - Don’t chase the mirage of iron-fortified rice - https://indianexpress.com/article/opinion/columns/rice-fortification-programme-iron-anaemia-7470938/