May 13, 2017
By Lovisa Tenberg and Konstantin Sietzy
Cross-posted to the Oxford Prioritisation Project blog. We're centralising all discussion on the Effective Altruism forum. To discuss this post, please comment here.
Summary: In 2016 James Snowden of the Centre for Effective Altruism built a quantitative model estimating the impact of StrongMinds. In order to measure our uncertainty about the estimate, we built a detailed, annotated translation of the model in Guesstimate (a “spreadsheet for things that aren’t certain”) which can be found here. This post is acts as an appendix to the Guesstimate model.
Improving mental health (MH) in lower and middle income countries (LMIC) as a highly effective cause has received increasing attention in the Effective Altruism community over the past year (GWWC has written on it here and here; Harvard EA has written on it here; and Givewell staff members donated to it in 2016). It is vast in scale and mostly neglected in LMICs, but tractability remains somewhat of a black box.
MH is vast in scale, amounting for 7.4% of the Global Burden of Disease, but 37% of the Global Burden of Non-Communicable Disease. Importantly, the outlook is worsening: DALYs due to mental illness grew by 38% between 1990 and 2010, and are expected to continue on this trajectory.
MH is neglected both in-country and in the international community. According to Victoria de Menil writing for the Centre of Global Development, “one third of LMIC do not have a designated budget for mental health… and among those that do, the average expenditure on mental health in low-income countries is 0.5% of the total health budget.” She contrasts the treatment gap in Nigeria – 90% across all diagnosable mental disorders – with an average of 32% for schizophrenia to 56-57% for depression and anxiety disorders in European countries. The international donor community affords MH improvement disproportionately little attention given its scale: only 0.7% of international donor funding is directed towards it, and hardly any NGOs work directly on MH, especially not at the level of widely scalable interventions.
On tractability, the evidence is out. Most existing scholarship on the effectiveness of MH interventions exists in a high-income country context, and has little focus on the cost-effectiveness of interventions. Yet it is increasingly on the map of global health professionals and national governments, having been included in the January 2016 Sustainable Development Goals after high-profile campaigning. In addition, King’s College and the London School of Hygiene and Tropical Medicine are conducting 40+ trials into various components of the effectiveness of MH interventions as part of their Centre for Global Mental Health.
StrongMinds treats women with depression in Uganda. The organisation recruits community mental health facilitators to treat groups of 10-12 women using Interpersonal Group Therapy (IPT-G) over a course of 8-12 weeks. StrongMinds is committed to impact measurement and scalability: internal figures suggest that 80% of patients are depression-free at the end of treatment, with the effect being stable in follow-ups. They also claim a 67% decrease in unemployment. StrongMinds aims to scale to 100,000 patients treated by 2019 and 2m by 2025. StrongMinds has been lauded as a highly transparent conversation in private conversations with GWWC staff members, and is led by pedigreed staff in the global health sector (their CEO is an ex-employee of PSI).
The model is based on a 2016 model created by James Snowden for CEA. It is mainly based on StrongMinds organisational data and internal impact assessment results. A detailed, annotated translation of the model in Guesstimate can be found here.
Key inputs are i) StrongMinds data on expenses and patients treated; ii) converted DALY weights of 1 point reductions on the PHQ-9 depression scale used to measure StrongMinds impact; iii) trajectory of treatment effect of IPT-G for depression over time, adapted from research by Rebecca Reay et al. (2012). The model outputs data for both the year 2016, and a 2019 estimate ("2019E"), based on StrongMinds projections of their expenses and number of patients treated.
StrongMinds per-patient costs were assessed using YTD mid-2016 cost and patient treated metrics, arriving at a cost per patient of $206. StrongMinds hopes to scale to having treated 100,000 patients by 2019 at a total cost of $6.6m, reducing their per-patient cost over time by roughly 2/3rds to $66.
StrongMinds measures impact on the 27-point, linear PHQ-9 scale. To convert PHQ-9 impact to DALYs averted, Global Burden of Disease DALY-weighting of most severe depression (0.658) was divided by PHQ-9 points-weighting of most severe depression (27) to render 0.024 DALYs averted per PHQ-9 point reduced.
As StrongMinds’ impact assessment only measures absolute effect, the model proxies counterfactual impact by relying on Reay et al. who compare the trajectory of IPT-G patients’ depression with an untreated control group. The model extrapolates Reay’s comparison over a ten year span to arrive at a total multiplier of original impact over time that is applied to the baseline treatment effect.
The model suffers from three key uncertainties. Firstly, there is an established case that preference-based measurements such as DALYs may underrate the badness of depression. Secondly, the trajectory of treatment effect vs a counterfactual is proxied through research on postpartum depression only. Other types of depression may not exhibit a declining counterfactual over time. Simultaneously, other types of depression may be less amenable to IPT-G. Finally, data (and the 2019E) is solely based on StrongMinds internal assessments, suggesting caution.