[Not answering on behalf of HLI, but I am an HLI employee]
Hi Michal,
We are interested in exploring more systematic solutions to aligning institutions with wellbeing. This topic regularly arises during strategic conversations.
Our aim is to eventually influence policy, for many of the reasons you mention. But we’re currently focusing on research and philanthropy. This is because there’s still a lot we need to learn about how to measure and best improve wellbeing. But before we attempt to influence how large amounts of resources are spent, I think...
I disagree because I think writing text to indicate a sentiment is a stronger signal than pressing a button. So while it’s somewhat redundant, it adds new information IMO.
As a writer, I pay attention to these signals when processing feedback.
Hi again Jason,
When we said "Excluding outliers is thought sensible practice here; two related meta-analyses, Cuijpers et al., 2020c; Tong et al., 2023, used a similar approach" -- I can see that what we meant by "similar approach" was unclear. We meant that, conditional on removing outliers, they identify a similar or greater range of effect sizes as outliers as we do.
This was primarily meant to address the question raised by Gregory about whether to include outliers: “The cut data by and large doesn't look visually 'outlying' to me....
Hi Jason,
“Would it have been better to start with a stipulated prior based on evidence of short-course general-purpose[1] psychotherapy's effect size generally, update that prior based on the LMIC data, and then update that on charity-specific data?”
1. To your first point, I think adding another layer of priors is a plausible way to do things – but given the effects of psychotherapy in general appear to be similar to the estimates we come up with[1] – it’s not clear how much this would change our estimates.
There are probably two ...
Hi Victor,
Our updated operationalization of psychotherapy we use in our new report (page 12) is
"For the purposes of this review, we defined psychotherapy as an intervention with a structured, face-to-face talk format, grounded in an accepted and plausible psychological theory, and delivered by someone with some level of training. We excluded interventions where psychotherapy was one of several components in a programme."
So basically this is "psychotherapy delivered to groups or individuals by anyone with some amount of training".
Does that...
They only include costs to the legal entity of StrongMinds. To my understanding, this includes a relatively generous stipend they provide to the community health workers and teachers that are "volunteering" to provide StrongMinds or grants StrongMinds makes to NGOs to support their delivery of StrongMinds programs.
Note that 61% of their partnership treatments are through these volunteer+ arrangements with community health workers and teachers. I'm not too worried about this since I'm pretty sure there aren't meaningful additional costs to consider. t...
Hi Nick,
Good question. I haven't dug into this in depth, so consider this primarily my understanding of the story. I haven't gone through an itemized breakdown of StrongMinds costs on a year by year basis to investigate this further.
It is a big drop from our previous costs. But I originally did the research in Spring 2021, when 2020 was the last full year. That was a year with unusually high costs. I didn't use those costs because I assumed this was mostly a pandemic related aberration, but I wasn't sure how long they'd keep the more expensive practi...
Hi Jason,
The bars for AMF in Figure 2 should represent the range of cost-effectiveness estimates that come from inputting different neutral points, and for TRIA the age of connectedness.
This differs from the values given in Table 25 on page 80 because, as we note below that table, the values there are based on assuming a neutral point of 2 and an TRIA age of connectedness of 15.
The bar also differs for the range given in Figure 13 on page 83 because the lowest TRIA value has an age of connectivity of 5 years, where in Figure 2 (here) we a...
Neat work. I wouldn't be surprised if this ends up positively updating my view on the cost-effectiveness of advocacy work.
What's your take on possibility someone could empirically tackle a related issue we also tend to do a lot of guessing at -- the likelihood of $X million spent advocacy in a certain domain leading to reform.
The prospect of a nuclear conflict is so terrifying I sometimes think we should be willing to pay almost any price to prevent such a possibility.
But when I think of withdrawing support for Ukraine or Taiwan to reduce the likelihood of nuclear war, that doesn't seem right either -- as it'd signal that we could be threatened into any concession if nuclear threats were sufficiently credible.
How would you suggest policymakers navigate such terrible tradeoffs?
How much do you think the risk of nuclear war would increase over the century if Iran acquired nuclear weapons? And what measures, if any, do you think are appropriate to attempt to prevent this or other examples of nuclear proliferation?
Joel, I’m highly confident Iran will not acquire a nuclear weapon. The U.S. and Israel have exquisite intelligence on the Iranian nuclear program, which has been a high priority for decades. Should Supreme Leader Ali Khamenei change his policy and pursue a nuclear weapon, we would know.
During my time in government I was involved in convincing the Israeli government not to launch a military strike against Iranian nuclear facilities. I was also involved in developing the military capabilities needed if Iran did opt for nuclear weapons, and these capabilit...
note that a large portion of Somaliland appears occupied by rebels at the moment. But other than that it has indeed been much more peaceful.
“Thank you for the comment. There’s a lot here. Could you highlight what you think the main takeaway is? I don’t have time to dig into this at present, so any condensing would be appreciated. Thanks again for the time and effort.” ??
I believe that large tech companies are, on average, more efficient at converting talent into market cap value than small companies or startups are. They typically offer higher salaries, for one.
This may be true for market cap, but let's be careful when translating this to do-goodery. E.g., wages don't necessarily relate to productivity. Higher wages could also reflect higher rents, which seems plausibly self-reinforcing by drawing (and shelving) innovative talent from smaller firms. A quote from a recent paper by Akcigit and Goldschlag (2023) is suggestiv...
I don't think this is right- "Russia" doesn't make actions, Vladimir Putin does; Putin is 70, so he seems unlikely to be in power once Russia has recovered from the current war; there's some evidence that other Russian elites didn't actively want the war, so I don't think it's right to generalize to "Russia".
Even if it was true that many elites were anti-war before the invasion, I think the war has probably accelerated a preexisting process of ideological purification. So even when Putin kicks the can, I think the elites will be just as likely to say "We d...
Hah! Yeah, stepping back, I think these events are a distraction for most people. Especially if they worsen one's mental health. For me, reflecting on the war makes me feel so grateful and lucky to live where I do.
Another reason to pay attention is when it seems like it could shortly and sharply affect the chances of catastrophe. At the beginning of the war, I kept asking myself, "At what probability of nuclear war should I: make a plan, consider switching jobs, move to Argentina, etc." But I think we've moved out of the scary zone for a while.
Fair jabs, but the PRC-Taiwan comparison was because it was the clearest natural experiment that came to mind where different bits of a nation (shared language, culture, etc.) were somewhat randomly assigned to authoritarianism or pluralistic democracy. I'm sure you could make more comparisons with further statistical jiggery-pokery.
The PRC-Taiwan comparison is also because, imagining we want to think of things in terms of life satisfaction, it's not clear there'd be a huge (war-justifying) loss in wellbeing if annexation by the PRC only meant a rela...
I agree that the agency of newer NATO members (or Ukraine) has been neglected. Still, I don't think this was a primary driver of underestimating Ukraine's chances -- unless I'm missing what "agency" means here.
I assume predictions were dim about Ukraine's chances at the beginning of the war primarily because Russia and the West had done an excellent job of convincing us that Russia's military was highly capable. E.g., I was disconcerted by the awe/dread with which my family members in the US Army spoke about Russian technical capabilities across mult...
What’s the track record of secular eschatology?
A recent SSC blog post depicts a dialogue about Eugenics. This raised the question: how has the track record been for a community of reasonable people to identify the risks of previous catastrophes?
As noted in the post, at different times:
While there are some Metaculus questions that ask for predictions of the actual risk, the ones I selected are all conditional of the form, "If a global catastrophe occurs, will it be due X". So they should be more comparable to the RP question "Which of the following do you think is most likely to cause human extinction?"
I know this wasn't the goal, but this was the first time I'd seen general polls of how people rank existential risks, and I'm struck by how much the public differs from Rationalists / EAs (using Metaculus and Toby as a proxy). [1]
Risk | Public (RP) | Metaculus | Difference |
Nukes | 42% | 31% | -11% |
Climate | 27% | 7% | -20% |
Asteroid | 9%[2] | ~0% (Toby Ord) | -9% |
Pandemic | 8% | Natural: 14%, Eng: 28% | 6- 20% |
AI | 4% | 46% | 42% |
In general, i agree politeness doesn’t require that — but id encourage following up in case something got lost in junk if the critique could be quite damaging to its subject.
In case it’s not obvious, the importance of previewing a critique also depends on the nature of the critique and the relative position of the critic and the critiqued. I think those possibly “Punching down” should be more generous and careful than those “punching up”.
The same goes for the implications of the critique “if true”, whether it’s picking nits or questioning whether the organisation is causing net harm.
That said, I think these considerations only make a difference between waiting one or two weeks for a response and sending one versus several emails to a couple of people if there’s no response the first time.
Hi Alex, I’m heartened to see GiveWell engage with and update based on our previous work!
[Edited to expand on takeaway]
My overall impression is:
Hi John, it’s truly a delight to see someone visually illustrate our work better than we do. Great work!
Great piece. Short and sweet.
Given the stratospheric karma this post has reached, and the ensuing likelihood it becomes a referenced classic, I thought it'd be a good time to descend to some pedantry.
"Scope sensitivity" as a phrase doesn't click with me. For some reason, it bounces off my brain. Please let me know if I seem alone in this regard. What scope are we sensitive to? The scope of impact? Also some of the related slogans "shut up and multiply" and "cause neutral" aren't much clearer. "Shut up and multiply" which seems slightly offputti...
Jason,
You raise a fair point. One we've been discussing internally. Given the recent and expected adjustments to StrongMinds, it seems reasonable to update and clarify our position on AMF to say something like, "Under more views, AMF is better than or on par with StrongMinds. Note that currently, under our model, when AMF is better than StrongMinds, it isn't wildly better.” Of course, while predicting how future research will pan out is tricky, we'd aim to be more specific.
A high neutral point implies that many people in developing countries believe their lives are not worth living.
This isn't necessarily the case. I assume that if people described their lives as having negative wellbeing, this wouldn't imply they thought their life was not worth continuing.
...2. I don't think 38% is a defensible estimate for spillovers, which puts me closer to GiveWell's estimate of StrongMinds than HLI's estimate of StrongMinds.
I wrote this critique of your estimate that household spillovers was 52%. That critique had three parts. The third part was an error, which you corrected and brought the answer down to 38%. But I think the first two are actually more important: you're deriving a general household spillover effect from studies specifically designed to help household members, which would lead to an overestimate.
My intuition, which is shared by many, is that the badness of a child's death is not merely due to the grief of those around them. Thus the question should not be comparing just the counterfactual grief of losing a very young child VS an [older adult], but also "lost wellbeing" from living a net-positive-wellbeing life in expectation.
I didn't mean to imply that the badness of a child's death is just due to grief. As I said in my main comment, I place substantial credence (2/3rds) in the view that death's badness is the wellbeing lost. Again, this my view n...
To be clear on what the numbers are: we estimate that group psychotherapy has an effect of 10.5 WELLBYs on the recipient's household, and that the death of a child in a LIC has a -7.3 WELLBY effect on the bereaved household. But the estimate for grief was very shallow. The report this estimate came from was not focused on making a cost-effectiveness estimate of saving a life (with AMF). Again, I know this sounds weasel-y, but we haven't yet formed a view on the goodness of saving a life so I can't say how much group therapy HLI thinks is preferable avertin...
I'd point to the literature on time lagged correlations between household members emotional states that I quickly summarised in the last installment of the household spillover discussion. I think it implies a household spillover of 20%. But I don't know if this type of data should over- or -underestimate the spillover ratio relative to what we'd find in RCTs. I know I'm being really slippery about this, but the Barker et al. analysis stuff so far makes me think it's larger than that.
I find nothing objectionable in that characterization. And if we only had these three studies to guide us then I'd concede that a discount of some size seems warranted. But we also have A. our priors. And B. some new evidence from Barker et al. Both of point me away from very small spillovers, but again I'm still very unsure. I think I'll have clearer views once I'm done analyzing the Barker et al. results and have had someone, ideally Nathanial Barker, check my work.
[Edit: Michael edited to add: "It's not clear any specific number away from 0 could ...
[Michael's response below provides a shorter, less-technical explanation.]
Alex’s post has two parts. First, what is the estimated impact of StrongMinds in terms of WELLBYs? Second, how cost-effective is StrongMinds compared to the Against Malaria Foundation (AMF)? I briefly present my conclusions to both in turn. More detail about each point is presented in Sections 1 and 2 of this comment.
GiveWell estimates that StrongMinds generates 1.8 WELLBYs per treatment (17 WELLBYs per $100...
Thank you for this detailed and transparent response!
I applaud HLI for creating a chart (and now an R Shiny App) to show how philosophical views can affect the tradeoff between predominately life-saving and predominately life-enhancing interventions. However, one challenge with that approach is that almost any changes to your CEA model will be outcome-changing for donors in some areas of that chart. [1]
For example, the 53-> 38% correction alone switched the recommendation for donors with a deprivationist framework who think the neutral point is ove...
Zooming out a little: is it your view that group therapy increases happiness by more than the death of your child decreases it? (GiveWell is saying that this is what your analysis implies.)
(EDITED)
Is this (other than 53% being corrected to 38%) from the post accurate?
...Spillovers: HLI estimates that non-recipients of the program in the recipient’s household see 53% of the benefits of psychotherapy from StrongMinds and that each recipient lives in a household with 5.85 individuals.[11] This is based on three studies (Kemp et al. 2009, Mutamba et al. 2018a, and Swartz et al. 2008) of therapy programs where recipients were selected based on negative shocks to children (e.g., automobile accident, children with nodding syndrome, children with psych
Joel from HLI here,
Alex kindly shared a draft of this report and discussed feedback from Michael and I more than a year ago. He also recently shared this version before publication. We’re very pleased to finally see that this is published!
We will be responding in more (maybe too much) detail tomorrow. I'm excited to see more critical discussion of this topic.
I'd assume that 1. you don't need the whole household, depending on the original sample size, it seems plausible to randomly select a subset of household members [1](e.g., in house A you interview recipient and son, in B. recipient and partner, etc...) and 2. they wouldn't need to consent to participate, just to be surveyed, no?
If these assumptions didn't hold, I'd be more worried that this would introduce nettlesome selection issues.
I recognise this isn't necessarily simple as I make it out to be. I expect you'd need to be more careful w
Given that this post has been curated, I wanted to follow up with a few points I’d like to emphasise that I forgot to include in the original comment.
Neat! Cover jacket could use a graphic designer in my opinion. It's also slotted under engineering? Am I missing something?
Dear Srdjan,
I think we do address the potential for negative impacts. As we say in section 2.2 (and elaborate on in Appendix B.3:
"From 11 studies we estimate that a 1% increase in immigrants as a share of the population is associated with a (non-significant) decrease of -0.004 SDs of SWB (or -0.008 WELLBYs) for the native population."
Additionally, we have a subsection (3.2) called "risk of backlash effects". Again, these may not be the concerns you have in mind, but to say we're only mentioning positive effects is wrong. We mention throughout t...
James courteously shared a draft of this piece with me before posting, I really appreciate that and his substantive, constructive feedback.
1. I blundered
The first thing worth acknowledging is that he pointed out a mistake that substantially changes our results. And for that, I’m grateful. It goes to show the value of having skeptical external reviewers.
He pointed out that Kemp et al., (2009) finds a negative effect, while we recorded its effect as positive — meaning we coded the study as having the wrong sign.
What happened is that MH outcomes are often "hi...
Given that this post has been curated, I wanted to follow up with a few points I’d like to emphasise that I forgot to include in the original comment.
Strong upvote for both James and Joel for modeling a productive way to do this kind of post -- show the organization a draft of the post first, and give them time to offer comments on the draft + prepare a comment for your post that can go up shortly after the post does.
The guess is based on a recent (unpublished and not sure I can cite) survey that I think did the best job yet at eliciting people's views on the neutral point in three countries (two LMICs).
I agree it's a big ask to get people to use the exact same scales. But I find it reassuring that populations who we wouldn't be surprised as having the best and worst lives tend to rate themselves as having about the best and worst lives that a 0 to 10 scale allows (Afghanis at ~2/10 and Finns at ~8/10.
That's not to dismiss the concern. I think it's plausible that there...
Trying to hold onto the word “eugenics” seems to indicate an unrealistically optimistic belief in people’s capacity to tolerate semantics. Letting go is a matter of will, not reason.
E.g., I pity the leftist who thinks they can, in every conversation with a non-comrade, explain the difference between the theory of a classless society, the history of ostensibly communist regimes committing omnicide, and the hitherto unrealised practice of “real communism” (outside of a few scores of 20th-century Israeli villages and towns). To avoid the reverse problem...
A note on the "positive utility" bit. I am very uncertain about this. We don't really know where on subjective wellbeing scales people construe wellbeing to go from positive to negative. My best bet is around 2.5 on a 0 to 10 scale. This would indicate that ~18% of people in SSA or South Asia have lives with negative wellbeing if what we care about is life satisfaction (debatable). For the world, this means 11%, which is similar to McAskill's guess of 10% in WWOTF.
And insofar as happiness is separate from life satisfaction. It's very rare for a count...
I haven't downvoted or read the post, but one explanation is the title "You're probably a eugenicist" seems clickbaity and aimed at persuasion. It reads as ripe for plucking out of context by our critics. I immediately see it cited in the next major critique published in a major news org: "In upvoted posts on the EA forum, EAs argue they can have 'reasonable' conversations about eugenics."
One idea for dealing with controversial ideas is to A. use a different word and or B. make it more boring. If the title read something like, "Most people favor selecting for valuable hereditary traits." My pulse would quicken less upon reading.
I dont see this as much of an update. Mutual inspections under the treaty haven‘t taken place for a year, it’s basically already been suspended since the invasion. I would be more concerned if he formally withdrew, but he didn’t even do that.
In retrospect, I think my reply didn't do enough to acknowledge that A. using a different starting value seems reasonable and B. this would lead to a much smaller change in cost-effectiveness foor deworming. While very belated, I'm updating the post to note this for posterity.
I agree that advocacy for high skilled immigration is more likely to succeed, and that the benefits would probably come more from technological and material progress. The problem is we currently aren't prepared to try and estimate the benefits of these society and world wide spillover effects.
Maybe we will return to this if (big if) we explore policies that may cost-effectively increase GDP growth (which some argue is = tech progress in the long run?), and through that subjective wellbeing [1] .
Regarding Malengo, I asked Johannes a fe...
Hi David, I'm excited about this! It certainly seems like a step in the right direction. A few vague questions that I'm hoping you'll divine my meaning from:
Keep up the good struggle my dear fish loving friends.