Recently, when I was assessing my career options in terms of impact I prepared a rough estimate of the social impact of psychotherapy. I haven't encountered such an estimate before - other than in a google sheet calculation that I received from an fellow EA via FB. I thought this might be helpful to people who are interested of doing a more comprehensive review of mental health as a cause.
Introduction
This is an attempt to roughly estimate the impact of a psychotherapist in units of QALYs added per therapist-hour. One QALY corresponds to value of one year lived in perfect health. This estimate is an upper bound for the social impact because it does not take into account the effects of diminishing returns and replaceability. Similar estimate has been earlier compiled by Lynette Bye [1].
Estimate #1
Following data are from a study on long-term cost-effectiveness of CBT in treating depression [2,3]:
- Mean annualized QALYs in intervention group 0.596 vs. 0.544 control group (usual care including antidepressants)
- Follow-up 3-5 years after intervention, on average 40 months after therapy ended
- Number of CBT sessions received: 12 – 18 (duration 50-60min)
Using these data is possible to estimate additional QALYs gained per therapist-hour:
- Additional QALYs: 0.596 – 0.544 = 0.052 for, on average 40/12 = 3.33 years resulting in 0.052*3.33 = 0.173 QALYs gained.
- 15 sessions lasting on average 55 minutes means 15*55/60 = 13.75 hours
- The resulting estimate is 0.173/ 13.75 = 0.0126 QALY/therapist-hour
Estimate #2
A second study on cost-effectiveness of interventions for social anxiety disorder [4] (this is one of the studies included in earlier estimate [1]) contains the following data:
- Best CBT treatment (ICBT, C&W) in resulted in average 3.75 QALYs gained vs. waitlist average 3.37 QALYs gained
- Best treatment (ICBT, C&W) included 14 individual sessions, 90 minutes each, resulting in 21 therapist-hours per intervention
These data result in an estimate of (3.75-3.37) / 21 = 0.0181 QALY/therapist-hour.
Conclusion
The two studies resulted in estimates in the range of 0.012-0.018 QALY/therapist-hour. This is in the same ballpark as the result of earlier calculation [1]: roughly 0.157 QALY / 10 sessions = 0.016 / session.
Even though these calculations are based on two, in my opinion high quality studies, the estimate is rough because the basis is just two studies whose QALY estimates contain a fair bit of uncertainty. Secondly, it is unclear how long the effects of therapy persist. Based on these studies, they might persist for a long time (5 years) but it is unclear what is the average effect. Thirdly, improved mental health might have positive secondary network effects for the well-being of patient’s immediate social circle, e.g. children, spouse and close relatives. These hypothetical network effects are not included in the estimate.
To assess the impact of psychotherapy alone, it is better to compare psychotherapy to treatment as usual without psychotherapy (i.e. pharmacotherapy) rather than no treatment. Estimate #1 compared psychotherapy to usual care whereas estimate #2 compared it to waitlist which makes estimate #1 somewhat more credible. Overall, taking into account the high level of uncertainty involved, I estimate that the impact of psychotherapy is in the range of 0.005-0.03 QALY/therapist-hour for mental health problems with good treatment response to therapy such as depression and social anxiety.
Notes regarding impact over career
By estimating the number of therapist-hours a psychotherapist works over his or her career it is possible to gauge the overall impact. For example, assuming 20 therapist-hours per week, that is 27 45-minute sessions per week, for 45 weeks per year for 30 years one gets an estimate of 20*45*30 * [0.005 – 0.03] = 135 – 810 QALYs or 5 – 27 lives saved using conversion rate of 30 QALYs = “1 life saved” [5].
Sources
[1] Counseling impact model: some estimates compiled by Lynette Bye
[2] Wiles, N. J., Thomas, L., Turner, N., Garfield, K., Kounali, D., Campbell, J., … & Williams, C. (2016). Long-term effectiveness and cost-effectiveness of cognitive behavioural therapy as an adjunct to pharmacotherapy for treatment-resistant depression in primary care: follow-up of the CoBalT randomised controlled trial. The Lancet Psychiatry, 3(2), 137-144.
[3] Wiles, N., Thomas, L., Abel, A., Ridgway, N., Turner, N., Campbell, J., … & Kuyken, W. (2013). Cognitive behavioural therapy as an adjunct to pharmacotherapy for primary care based patients with treatment resistant depression: results of the CoBalT randomised controlled trial. The Lancet, 381(9864), 375-384.
[4] Mavranezouli, I., Mayo-Wilson, E., Dias, S., Kew, K., Clark, D. M., Ades, A. E., & Pilling, S. (2015). The cost effectiveness of psychological and pharmacological interventions for social anxiety disorder: a model-based economic analysis. PloS one, 10(10), e0140704.
[5] World bank uses 30 DALYs = “1 life saved”. Here, 1 DALY is assumed to equal 1 QALY.
Great, thanks!
To broaden the analysis I think correcting for an implementation bias is useful. Fidelity to the protocol by psychotherapists is often way lower in real life than in research studies. This could make the average numbers more pessimistic, but the added value of a psychotherapist being aware of those cognitive impediments way higher, and possibly a more interesting career option (training and supervising younger therapists, lobbying for evidence based psychotherapy). But that just might be a self-serving bias speaking, the recent meta-analysis by Cuijpers made me doubt if I want to continue my work as a psychotherapist.
Yes, I agree: probably much of the therapy given is not given according to the protocol and that means the average effectiveness is likely lower than the numbers in the studies indicate. In many cases, I think this might not be due to the psychotherapists or therapists themselves but the organization which they work in, e.g. crowded outpatient clinics where the policy is to meet each client in every 3 weeks or in order to not to make the queues to treatment appear so long.
I think think there might be potential for big impact for somebody with clinical background who is willing to advocate long-term for systemic change within mental health care and psychotherapy: optimal treatment protocols (best value per therapy-hour or so), triage, adherance to protocols etc.
Nice initial analysis to get some ballpark numbers on this. You probably already considered this and I have no insight into the profession but I would imagine therapists often work with patients having mental health problems with lower or very little treatment response compared to depression/anxiety.
You also wrote that the estimate is not replaceability-adjusted but what is your basic estimate about this factor? Similar to the replaceability of medical doctors? My intuition is that the amount of therapists is more limited by the number of therapist offices than by the supply of eager psych students. But I also haven't had a detailed look into Lynette Byes model yet to see if it's mentioned there.
Thanks. Yes, many therapists work with people who have mental health problems with weaker treatment response to psychotherapy such as bipolar disorder, psychotic disorders and various personality disorders. This lowers the average impact or effectiveness of psychotherapy over the whole population treated
I haven't tried to make explicit estimate of replaceability. My baseline estimate would be the same number as used for doctors (0.6 in 80000 hours article) because both occupations are highly skilled.
I think the replaceability is dependent on the specific country system and conditions, ie. the educational system that enables a person to become a psychotherapist and the labour market. In Finland, where I live, anybody who is a licensed psychotherapist can set up their own therapist office (by this I mean a private practice). Considering Finland, my (considerably uncertain) hunch is that number of licensed psychotherapists is most limited by the number of high quality applicants (you need a relevant masters degree and several years of clinical experience to apply) to the psychotherapy training programs. But even if you add a psychotherapist to the pool of people willing and able to work as a psychotherapist, you will not very likely be adding a one full-time therapist worth of therapist-hours to the pool of therapy given because some people (likely those less skilled and/or those who have graduated from less popular schools of psychotherapy) will probably be working less.
Super interesting, and I was just having this conversation recently. There's one issue I have with the analysis of psycholotherapy (assuming we even get a control group, which few studies do). The data points we're using to calculate effectiveness is self-reported. In other words, we have no external method of evaluating the actual positive impact impartially, only as it was experienced by the participants. Sunk-cost fallacy, the Hawthorne Effect, etc, could inspire truly believed but ultimately inaccurate reporting.