Hide table of contents

This shallow review was written by SoGive. SoGive is an organisation which provides services to donors to help them to achieve high impact donations.

This is a very quick, rough model of the cost-effectiveness of promoting clean cookstoves in the developing world. It suggests that:

- If a clean cookstove intervention is successful, it may have roughly the same ballpark of cost-effectiveness as a GiveWell-recommended charity

- C.90% of the impact comes from directly saving lives, in a model which reflected saving lives and climate change impact

This is very much not intended to be a final, polished analysis of the topic. In particular, in order to make this a quick, bite-sized piece of analysis, a number of important assumptions were made, notably:

  • We did not carefully review how easy/tractable it is to get cookstoves into homes and *actually maintained and used*. We are aware that J-PAL has reviewed this question some time ago and concluded that this is difficult. More recently IDinsight considered this topic and found that it can be made to work. The IDinsight analysis only came to our attention after this short piece was already mostly drafted, and we may update our views after we have considered the IDinsight piece more carefully.
  • The analysis assumes that clean cookstoves really are clean (i.e. that all the premature deaths from indoor air pollution would go away if clean cookstoves were universally adopted). (Note: after creating this model, we observed that this ProPublica article seems to suggest that this assumption is unlikely to be true. As this was a quick, shallow review, we have not yet worked out how much the model should be adjusted for this, if at all)
  • A number of factors were omitted from the model which are likely to understate the true impact, notably: cost savings to family, differing amounts of time spent on cooking (including fuel collection time), morbidity impact of replacing unclean cookstoves
  • We also omitted the impact on malaria, which is appears to be a small (or possibly close to zero) effect, but if anything goes the other way (i.e. is a disadvantage of replacing biomass cookstoves with clean cookstoves)
  • Lives saved equivalents for climate change were calculated using the $417 per tonne of CO2eq social cost of carbon from a Nature Climate Change paper, and then converted into lives saved equivalents using the moral weights in the GiveWell cost-effectiveness analysis.
  • Another factor not considered in this analysis is whether donations are the right way to fund the intervention. In general, where an intervention can be effectively be funded by investment, a donor should consider whether that is a better way to fund the organisation. It seems likely that this would be a relevant consideration here too, and may materially argue against donating to support this area of work.

The model

A rough cost-effectiveness model for replacing basic biomass cookstoves with improved cookstoves can be found in this spreadsheet, and is also set out at a high level in Appendix 2. https://docs.google.com/spreadsheets/d/1xqFz5Lhuc5x__SPrbuHgWtou289lF3oZrinnaoV7whI/edit#gid=0


Findings of the model

If we simply look at the lives saved benefit only, we get a cost per life saved equal to

· Cost of subsidising cookstoves / number of lives saved = $45bn/3.8m = $11,800 (3sf)

This $11,800 figure gives no credit to the CO2/climate benefits.

· Cost per life saved equivalent factoring in climate change effects = $10,700 (3sf)

The climate impact is modelled using the social cost of carbon from a recent paper in Nature Climate Change which gave a social cost of carbon of $417 per tonne. Having converted the tonnes of CO2 to $, this is then converted to lives saved using the same moral weights as used in the GiveWell CEA. This results in climate change making a relatively small difference compared to the direct air pollution effect on mortality. Note that the social cost of carbon is just based on the economic effects of climate change – other effects (e.g. elevated risk of conflict) are not modelled, so arguably the climate element of this model understates the true impact.

A typical cost per life saved equivalent for a GiveWell-recommended charity is roughly $2,000 (see this post for more http://thinkingaboutcharity.blogspot.com/2019/12/how-cost-effective-is-cost-effective.html) For GiveDirectly, the cost per life saved equivalent is closer to $20,000.

So cookstoves are a bit behind the most cost-effective charities, but not in totally the wrong ballpark.

However it's unclear whether a fuller analysis taking into account the factors listed at the beginning of this piece would still come to the same conclusion.


Appendix 1: Some rough notes from our background reading on the topic

· This 2013 article (Martin, Glass et al 2013) is a pretty decent overview (although no checks done on how much of it is out of date now) https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3672215/

· This WHO fact sheet has some handy data: https://www.who.int/en/news-room/fact-sheets/detail/household-air-pollution-and-health

· J-PAL has some useful papers.

o This paper finds that use declines over time https://www.povertyactionlab.org/evaluation/cooking-stoves-indoor-air-pollution-and-respiratory-health-india

o This paper explored the way that gender dynamics influenced decision-making in the household https://www.povertyactionlab.org/evaluation/demand-nontraditional-cookstoves-bangladesh

o This is also saying that use declines over time https://www.povertyactionlab.org/sites/default/files/publications/52%20Up%20in%20Smoke%20AEJ2016.pdf

o This is a nice short factsheet: https://www.povertyactionlab.org/sites/default/files/publications/2012.08.29-Cookstoves.pdf

· The World Bank has some useful papers

o http://documents.worldbank.org/curated/en/732691468177236006/pdf/632170WP0House00Box0361508B0PUBLIC0.pdf

o http://documents.worldbank.org/curated/en/164241468178757464/pdf/98664-REVISED-WP-P146621-PUBLIC-Box393185B.pdf

· This article seems to cover how the cleaner cookstoves compare with the old ones (I haven’t looked at this article properly yet) https://qz.com/1327615/why-does-the-global-alliance-for-clean-cookstoves-promote-fossil-fuels/

· It refers to this (longer) piece: https://www.propublica.org/article/cookstoves-push-to-protect-the-planet-falls-short


Some notes from an article by the Washington Post:

https://www.washingtonpost.com/opinions/these-cheap-clean-stoves-were-supposed-to-save-millions-of-lives-what-happened/2015/10/29/c0b98f38-77fa-11e5-a958-d889faf561dc_story.html

· About 3 billion of the world’s people burn wood, charcoal or dung in smoky open fires to cook their food and heat their homes.

· Each year, close to 4 million people die prematurely from illness attributable to household air pollution from inefficient cooking practices using polluting stoves paired with solid fuels and kerosene.

· Source: WHO https://www.who.int/en/news-room/fact-sheets/detail/household-air-pollution-and-health

· This JPAL study (Bailis, Dwivedi et al https://www.povertyactionlab.org/evaluation/demand-nontraditional-cookstoves-bangladesh) found that women have stronger preferences for improved stoves than their husbands, but lack the authority to make purchasing decisions. Their findings also suggest that marketing campaigns can prompt initial adoption of unfamiliar technologies like improved stoves, but are less effective in the long run as common experience with technologies grows. Interestingly the study noted that when women were initially asked about clean cookstoves on their own they were more positive about them than their husbands, but in the few months between order and delivery, their opinions on the topic had come more in line with that of their husbands.

· This JPAL study (Hanna et al 2016 https://www.povertyactionlab.org/sites/default/files/publications/52%20Up%20in%20Smoke%20AEJ2016.pdf) found that smoke inhalation initially falls when people get an improved cookstove, but that the effect disappears by year 2.

· This Nov 2014 World Bank study on Clean and Improved cooking in Sub-Saharan Africa (http://documents.worldbank.org/curated/en/164241468178757464/pdf/98664-REVISED-WP-P146621-PUBLIC-Box393185B.pdf) states that “So far, three decades of efforts to promote both modern fuels and improved biomass stoves have seen only sporadic success at scale in the region and globally”. However it also strikes an optimistic note, suggesting that with the right conditions, the coming years could serve as a turning point for the sector.

· A notable exception to the claims that it’s hard to get cookstoves to be effective was a government programme in China that got more than 100 million cookstoves into people’s homes (source: World Bank study dated 2011 http://documents.worldbank.org/curated/en/732691468177236006/pdf/632170WP0House00Box0361508B0PUBLIC0.pdf) China’s government is better able to dictate this sort of thing to its people than more democratic countries.

· Journalist Meera Subramanian visited a village in northern India that had been declared “smoke-free” after a non-profit distributed biomass cookstoves there. She found that women had stopped using the stoves because they didn’t like the design, or because the stoves broke, burned more wood (not less, as intended) or didn’t get foods hot enough. “I couldn’t find a single stove operating in a condition resembling what its designers intended,” she writes in her book “A river runs again”. The Appropriate Rural Technology Institute, which gave away the stoves, took a survey two years later and found that only 20% were still in use. “Why are they cheating us by giving us things which break so early?” one woman complained to the agency.

· Affordability remains a fundamental challenge. Dirtier biomass cookstoves sell for $25 or less, but more complex stoves which run on electricity or use liquid fuels typically cost more and require access to a steady and reliable source of fuel.



Appendix 2: Summary of the model

Benefits

· “In developing countries, about 730 million tons of biomass are burned each year, amounting to more than 1 billion tons of carbon dioxide (CO2) emitted into the atmosphere.” (Source: World Bank 2011 http://documents.worldbank.org/curated/en/732691468177236006/pdf/632170WP0House00Box0361508B0PUBLIC0.pdf)

· Clean cookstoves can reduce fuel use by 30-60% (source: clean cooking alliance https://www.cleancookingalliance.org/impact-areas/environment/index.html)

· WHO estimate of premature deaths from cooking over open fires increased from 1.9 million to 4.3 million (source: Martin, Glass, et al 2013 https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3672215/) However the 4.3 million is reported as 3.8 million in this WHO factsheet (https://www.who.int/en/news-room/fact-sheets/detail/household-air-pollution-and-health) so I will use 3.8 million.

· Not modelled: cost savings to family, differing amounts of time spent on cooking (including fuel collection time), morbidity impact of replacing unclean cookstoves, impact on malaria.

All of this doesn’t take into account the possibility that interventions to reduce exposure to indoor air pollution may increase exposure to mosquitoes. See Biran, Smith et al 2007 https://www.ncbi.nlm.nih.gov/pubmed/17888474/ for more on this, but Biran, Smith et al suggested that this probably wasn’t a worry, but it also wasn’t conclusive.

Costs

The cost side of the cost-benefit analysis is modelled by the cost of subsidising the purchase of an improved cookstove.

a) The cost of a new stove is more than $50 (using the example of Inyenyeri in Rwanda, as described in this Washington Post article https://www.washingtonpost.com/opinions/these-cheap-clean-stoves-were-supposed-to-save-millions-of-lives-what-happened/2015/10/29/c0b98f38-77fa-11e5-a958-d889faf561dc_story.html) and most customers will need two or three, so let’s say the cost is $150.

b) The initial willingness to pay for a quality Improved Cook Stove is often 20% - 50% of stove value, but can be increased with marketing and consumer education (Soruce: World Bank paper http://documents.worldbank.org/curated/en/164241468178757464/pdf/98664-REVISED-WP-P146621-PUBLIC-Box393185B.pdf)

c) Total number of people using solid (biomass) fuel is 3 billion (source: this World Bank paper http://documents.worldbank.org/curated/en/732691468177236006/pdf/632170WP0House00Box0361508B0PUBLIC0.pdf or this WHO factsheet https://www.who.int/en/news-room/fact-sheets/detail/household-air-pollution-and-health)

d) Let’s assume that the number of people fed per stove is 5 (source: guess)

e) So the total number of stoves to be replaced is 600 million (= (c) / (d))

f) Assuming that the willingness to pay is 50% (see (b)) and the cost is $150 (see (a)), this means that the cost is 50% × $150 × 600 million stoves = $45 billion. This assumes that the cost to implement the change is the cost of subsidising the remaining 50%.

Comments7
Sorted by Click to highlight new comments since: Today at 3:42 AM

You can track health impacts with HAPIT, the Household Air Pollution Intervention Tool . Use that directly instead of the assuming clean cookstoves eliminate the disease burden, which is far from the truth.

Clean cookstoves have a really high bar to clear to reduce the disease burden for several reasons:
1) Exposure is non-linear. The relative risk of dropping from 400 ug/m3 to 175 ug/m3 PM exposure is the same as going from 100 ug/m3 to 50 ug/m3. To reduce the disease burden to <1.5 requires exposure <50 PM ug/m3, which is very difficult for any biomass stove to accomplish.
2) For the disease burden to be reduced, nearly all stoves in a locality need to be replaced, otherwise the outdoor air will still be above the exposure threshold.
3) Stoves need to be used, used correctly, and maintained. The education and cultural habits to do this are very difficult to embed in a population.

For these reasons, clean cookstoves have historically been largely unsuccessful at reducing the disease burden. They are improving. See the "2019 Climate Action and Clean Cooking Co-benefits workshop presentations and discussions" presentation by the Clean Cooking Alliance. This group is setting standards and tiers for clean cookstoves to rank them on performance and targeting intervention locations based on many factors to determine where cookstoves will be most successful.

Even with optimistic assumptions of stove performance and uptake from HAPIT, it is likely that cookstoves will remain at least an order of magnitude more expensive than the best GiveWell recommended interventions for some time.

[Edit: fixed some spelling errors]

I didn't see it among your links, but GiveWell has an interim intervention report on this. Their summary is:

  • What is its evidence of effectiveness? Results from multiple randomized controlled trials (RCTs) suggest that distributions of clean cookstoves do not have clear evidence of effectiveness at reducing health problems attributable to air pollution. The evidence we have reviewed in our preliminary investigation finds limited impacts on women’s health and no clear impacts on children’s health under typical use. Distributions of clean cookstoves may have been less effective than expected due to implementation challenges, such as low compliance with using the replacement stoves and failure of the cleaner stoves to reduce air pollution sufficiently.
  • How cost-effective is it? We have not produced a cost-effectiveness model for clean cookstoves because we have not yet seen strong enough evidence to model a health benefit of the intervention.

I really appreciate these shallow reviews, as well as the comments providing additional sources!

Thanks for posting this shallow review! I strong-upvoted because I think it's really good for us to get more data on the Forum, even if it's shallow and flawed, as long as the author makes an effort to identify the flaws.

If you had a decent researcher who was willing to devote an extra week of work to the project (say, 25 focused hours plus check-ins with you), what are the questions you'd want them to cover?

Thanks for the encouragement. I think that aiming for a "perfect" write-up has been a barrier to publishing content, so I intend for us to publish more shallow reviews to address this.

To answer your question, I think the best focus areas would be the six bullet points highlighted near the start of the article, with a particular focus on the first two (are the stoves actually used, and are they actually clean?) and the last (what is the best way to fund this work?).

Also, we would further investigate the very useful comments made by MatthewDahlhausen (which seemed very useful and was upvoted by me) and look further at the GiveWell analysis as mentioned by cole_haus (I was aware of this, but had not had the capacity to review it properly)

Put a bed of rocks in a three-stone cookstove and fuel efficiency improves by a third.

Sun24 and its partners have trained six million women at a cost of about USD 0.05 per household. That is USD 0.004 per tonne of CO2e mitigated over five years. (Calculations)

I think this is important to investigate given the high mortality. I noticed that you ignored the savings in fuel. My understanding was that this could be quite significant and the stoves could pay for themselves either in saved fuel cost or saved opportunity cost of time from gathering the fuel. If this were true, you might be able to argue that the life savings and climate benefit came at zero cost. You would still have the issue that people are not willing to pay for them, perhaps because they have a high discount rate. Loans might ameliorate this.