On (1)
>people inflate their self-reports scores generally when they are being given treatment?
Yup, that's what I meant.
>Is there one or more studies you can point me to so I can read up on this, or is this a hypothetical concern?
I'm afraid I don't know this literature on blinding very well but a couple of pointers:
(i) StrongMinds notes "social desirability bias" as a major limitation of their Phase Two impact evaluation, and suggest collecting objective measures to supplement their analysis:
"Develop the means to negate ... (read more)
Excited to see your work progressing Michael!
I thought it might be useful to highlight a couple of questions I personally find interesting and didn't see on your research agenda. I don't think these are the most important questions, but I haven't seen them discussed before and they seem relevant to your work.
Writing this quickly so sorry if any of it's unclear. Not necessarily expecting an answer in the short term; just wanted to flag the questions.
(1) How should self-reporting bias affect our best guess of the effect size of therapy-ba... (read more)
I would deprioritise looking at BasicNeeds (in favour of StrongMinds). They use a franchised model and aren't able to provide financials for all their franchisees. This makes it very difficult to estimate cost-effectiveness for the organisation as a whole.
The GWWC research page is out of date (it was written before StrongMinds' internal RCT was released) and I would now recommend StrongMinds above BasicNeeds on the basis of greater levels of transparency, and focus on cost-effectiveness.
Thanks Holden. This seems reasonable.
A high impact foundation recently (and helpfully) sent me their grant writeups, which are a treasure trove of useful information. I asked them if I could post them here and was (perhaps naively) surprised that they declined.
They made many of the same points as you re: the limited usefulness of broad feedback, potential reputation damage, and (given their small staff size) cost of responding. Instead, they share their writeups with a select group of likeminded foundations.
I still think it would be much better if they mad... (read more)
I agree this seems relevant.
One slight complication is that donors to GWWC might expect a small proportion of people to renege on the pledge.
It seems like you're assuming that the GiveDirectly money would have gone only to the M-Pesa-access side of the (natural) experiment, but they categorized areas based on whether they had M-Pesa access in 2008-2010, not 2012-2014 when access was much higher.
Ah yes - that kind of invalidates what I was trying to do here.
I didn't notice that GiveWell had an estimate for this, and checking now I still don't see it. Where's this estimate from?
It came from the old GiveWell cost-effectiveness analysis excel sheet (2015). "Medians - cell V14". Act... (read more)
Thanks for this Jeff - a very informative post.
The study doesn't appear to control for cash transfers received through access to M-Pesa. I was thinking about how much of the 0.012 increase in ln(consumption) was due to GiveDirectly cash transfers.
Back of the envelope:
0.012 * 20m = 234,000 unit increases in ln(consumption)
GiveDirectly gave c.$9.5m in cash transfers between 2012-14 to people with access to M-Pesa. [1]
I agree. Although some forms of personal insurance are also rational. Eg health insurance in the US because the downside of not having it is so bad. But don't insure your toaster.
I agree that dmu over crop yields is perfectly rational. I mean a slightly different thing. Risk aversion over utilities. Which is why people fail the Allais pradadox. Rational choice theory is dominated by expected utility theory (exceptions Buchak, McClennen) which suggests risk aversion over utilities is irrational. Risk aversion over utilities seems pertinent here because most moral views don't have dmu of people's lives.
In normative decision theory, risk aversion means a very specific thing. It means using a different aggregating function from expected utility maximisation to combine the value of disjunctive states.
Rather than multiplying the realised utility in each state by the probability of that state occurring, these models apply a non-linear weighting to each of the states which depends on the global properties of the lottery, not just what happens in that state.
Most philosophers and economists agree risk aversion over utilities is irrational because it violates the... (read more)
We wanted to differentiate the website slightly from the eaglobal site while maintaining brand coherency so went for a slightly different shade of blue which feels a bit 'calmer'.
Not wedded to it though and may change back. Which do you prefer?
Thanks Michael - fixed now
Thanks Ian - agreed it doesn't look fantastic at the moment. We embedded it on the website at the last moment and it screwed with the formatting. We'll be working to improve how it looks over the next couple of weeks.
Thanks Austen. This is really helpful feedback.
Yes I agree. This is important but very hard to quantify. Of course the causal relationship goes both ways (poor physical health poor mental health) but it's probable that mental health disorders have worse downstream effects than most physical health problems (economic productivity, stigma, impact on carers, physical health). We tried to capture these qualitatively at the beginning of the report but could have been clearer that they weren't included in the cost-effectiveness calculations.
Thanks - this is
Eric - this is so great! Coincidentally, CEA has also been working on a very similar report which was completed last week. It's here: https://drive.google.com/open?id=0B551Ijx9v_RoZWlUUFVTYWZ6aTVCUDRDLTViVHVyQVpPWVNn
I've shot you an email. We should definitely discuss our conclusions.
1) Ah yes - thanks for pointing out. Probably has limited external validity for the strongminds model though (which is psychosocial treatment alone for most patients delivered by community health workers, with only the most serious cases referred to clinics for medication). The numbers come from the Chisholm (2015) WHO-CHOICE model. http://www.bmj.com/content/344/bmj.e609
2) Analysis is here https://docs.google.com/spreadsheets/d/1-lCC1zQHVZlJS8f9OfqhzcZTetHMxuMkW7nT75QDGhk/edit#gid=960072536
[This is quick and dirty but gives a rough indication of cost-effe... (read more)
Just to add to this. Acute schizophrenia is one of the worst health conditions on GBD13 DALY weightings (c.0.8). Severe depression is also one of the worst (c.0.65).
See http://www.thelancet.com/action/showFullTableImage?tableId=tbl2&pii=S2214109X15000698
So Michael - I agree it's very possible that mental health disorders are underweighted by DALY weightings because of the focusing illusion. But they are actually weighted quite highly at the moment. 10 years with severe depression is worth approximately 3.5 years of healthy life.
Hi Michael! As I said before, congrats on an interesting paper.
A few points on this comment:
1) DCP3 didn't have any cost-effectiveness figures for the StrongMinds intervention (interpersonal group therapy). Is the $1,000/DALY figure you mention related to primary care advice on alcohol use?
2) I'm currently writing a piece on mental health for a HNW donor and tried to model c-e of StrongMinds. I got c.$650/DALY reducing to $400/DALY as intervention scales. The biggest uncertainty in this estimate is the long term effects of psychosocial treatment as hardly ... (read more)
Thank you all for some great responses and apologies for my VERY late reply. This post was intended to 'test an idea/provoke a response' and there's some really good discussion here.
Bernadette,
Thank you for your very informative response. I must admit that my knowledge of EBM is much more limited than yours and is primarily Wikipedia-based.
The lines which particularly led me to believe that EBM favoured formal approaches rather than doctors' intuitions were:
"Although all medicine based on science has some degree of empirical support, EBM goes further, classifying evidence by its epistemologic strength and requiring that only the strongest types (coming from meta-analyses, systematic reviews, and randomized controlled trials) can ... (read more)
Thanks both for thoughtful replies and links.
I agree that it may be counterproductive to divide people who are answering the same questions into different camps and, on re-reading, that is how my post may come across. My more limited intention was to provide a (crude) framework through which we might be able to understand the disagreement.
I guess I had always interpreted (perhaps falsely) EA as making a stronger claim than 'we should be more reasonable when deciding how to do good'. In particular I feel that there used to be more of a focus on 'hard' rathe... (read more)
Very helpful summary, thanks