All of Simon_M's Comments + Replies

Claude's Summary:

Here are a few key points summarizing Will MacAskill's thoughts on the FTX collapse and its impact on effective altruism (EA):

  • He believes Sam Bankman-Fried did not engage in a calculated, rational fraud motivated by EA principles or long-termist considerations. Rather, it seems to have stemmed from hubris, incompetence and failure to have proper risk controls as FTX rapidly grew.
  • The fraud and collapse has been hugely damaging to EA's public perception and morale within the community. However, the core ideas of using reason and evidence to
... (read more)

If you click preview episode on that link you get the full episode. I also get the whole thing on my podcast feed (PocketCasts, not Spotify). Perhaps it's a Spotify issue?

Sorry, I edited while you posted. I see US as 1.44% * 27tn = ~400bn, which is the vast majority of charitable giving when I add in the rest of the countries Wikipedia lists and interpolate based on location for other biggish economies

0[comment deleted]5mo

Our friends estimate the cost at about $258 billion dollars to end extreme poverty for a year, and point out that this is a small portion of yearly philanthropic spending or [...]

Is that true?

Just ballparking this based on fractions of GDP given to charitable organisations (big overestimate imo), I get global giving at ~500bn/year. So I don't believe this is true

1
Alexander de Vries
5mo
You're right, I shouldn't have taken them at their word there, it's probably not a small portion of philanthropic spending. US phil spending is about 500 billion per year, and the US has the highest spending as % of GDP. Can't find sources for total worldwide spending online, but if I had to guess, it's about 1.5 trillion, in which case the named figure is some 20-ish percent of the total. Will edit post.

Now this is not… great, and certainly quite different from the data by tenthkrige. I'm pretty sure this isn't a bug in my implementation or due to the switch from odds to log-odds, but a deeper problem with the method of rounding for perturbation.

It's not particularly my place to discuss this, but when I replicated his plots I also found got very different results, and since then he shared his code with me and I discovered bug in it.

1
niplav
7mo
Interesting, good to know! I'm confused at this statement, though.

Basically it simulates the possible outcomes of all the other bets you have open.

How can I do that without knowing my probabilities for all the other bets? (Or have I missed something on how it works?)

3
Will Howard
8mo
It assumes the market probability is correct for all your other bets, which is an important caveat. This will make it more risk averse than it should be (you can afford to risk more if you expect your net worth to be higher in the future). It also assumes all the probabilities are uncorrelated, which is another important caveat. This one will make it less risk averse than it should be. I'm planning on making a version that does take all your estimates into account and rebalances your whole portfolio based on all your probabilities at once (hence mani–folio). This is a lot more complicated though, I decided not to run before I could walk. Also I think the simplicity of the current version is a big benefit, if you are betting over a fairly short time horizon and you don't have any big correlated positions then the above two things will just be small corrections.

Less concave = more risk tolerant, no?

Argh, yes. I meant more concave.

The point of this section is that since there are no good public estimates of the curvature of the philanthropic utility function for many top EA cause areas, like x-risk reduction, we don't know if it's more or less concave than a typical individual utility function. Appendix B just illustrates a bit more concretely how it could go either way. Does that make sense?

No, it doesn't make sense. "We don't know the curvature, ergo it could be anything" is not convincing. What you seem to think is "concrete" seems entirely arbitrary to me.

2
trammell
8mo
Hold on, just to try wrapping up the first point--if by "flat" you meant "more concave", why do you say "I don't see how [uncertainty] could flatten out the utility function. This should be in "Justifying a more cautious portfolio"?" Did you mean in the original comment to say that you don't see how uncertainty could make the utility function more concave, and that it should therefore also be filed under "Justifying a riskier portfolio"?

As Michael Dickens notes, and as I say in the introduction, I think the post argues on balance against adopting as much financial risk tolerance as existing EA discourse tends to recommend.

I appreciate you think that, and I agree that Michael has said he agrees, but I don't understand why either of you think that. I went point-by-point through your conclusion and it seems clear to me the balance is on more risk taking. I don't see another way to convince me other than putting the arguments you put forward into each bucket, weighting them and adding them up... (read more)

2
trammell
8mo
I can't speak for Michael of course, but as covered throughout the post, I think that the existing EA writing on this topic has internalized the pro-risk-tolerance points (e.g. that some other funding will be coming from uncorrelated sources) quite a bit more than the anti-risk-tolerance points (e.g. that some of the reasons that many investors seem to value safe investments so much, like "habit formation", could apply to philanthropists to some extent as well). If you feel you and some other EAs have already internalized the latter more than the former, then that's great too, as far as I'm concerned--hopefully we can come closer to consensus about what the valid considerations are, even if from different directions. Less concave = more risk tolerant, no? I think I'm still confused about your response on the second point too. The point of this section is that since there are no good public estimates of the curvature of the philanthropic utility function for many top EA cause areas, like x-risk reduction, we don't know if it's more or less concave than a typical individual utility function. Appendix B just illustrates a bit more concretely how it could go either way. Does that make sense?

but these arguments are not as strong as people claim, so we shouldn't say EAs should have high risk tolerance

I don't get the same impression from reading the post especially in light of the conclusions, which even without my adjustments seems in favour of taking more risk.

Yes, I thought your comment was great!

Can you elaborate on why you believe this? Are you talking specifically about global poverty interventions, or (EA-relevant) philanthropy in general? (I can see the case for global poverty[1], I'm not so sure about other causes.)

I was mosty thinking global poverty and health yes. I think it's still probably true for other EA-relevant philanthropy, but I don't think I can claim that here.

I'm also not clear on why you believe this, can you explain? (FWIW the claim in the parenthetical is probably false: on a cross-section across countries, GDP growth and equ

... (read more)

Just in the first few lines, we have a nitpick about the grammar of the title

I actually think this is substantial more than a nitpick. I doubt people are reading the whole of a 61(!) minute article and spot that the article doesn't support the title.

I'll grant the second point, I found critiquing this article extremely difficult and frustrating due to the structure and I think the EA forum would be much better if people wrote shorter articles, and it disappoints me that people seem to upvote without reading

1[anonymous]9mo
You say two things. 1. The conclusions doesn't seem to support that 2. I'm not sure it makes sense grammatically? "Against too much financial risk tolerance", "Against many arguments for financial risk tolerance"? I agree that (1) is substantial. (2) is not, and the response you give in the above comment doesn't provide reasons to think (2) is substantial. It was (2) I was commenting on. ETA: But perhaps now I'm nitpicking. I appreciate you acknowledging that you feel there was something to my other point.  ETA2: I won't reply further, because while I do stand by what I said, I also don't want to distract from what seems to me the more important discussion (about substance).

I think the article does support the title. By my read, the post is arguing:

  • many EAs claim that EAs should have high risk tolerance (as in, more risk tolerance than individual investors)
  • but these arguments are not as strong as people claim, so we shouldn't say EAs should have high risk tolerance

(I read "much" in the title as a synonym for "high", I personally would have used the word "high" but I have no problem with the title.)

I agree that short posts are generally better, I did find this post long (I reviewed it before publication and it took me abo... (read more)

Wow - this is a long post, and it's difficult for me to point out exact which bits I disagree with and which bits I agree with given it's structure. I'm honestly surprised it's so popular.

I also don't really understand the title. "Against much financial risk tolerance".

  1. The conclusions doesn't seem to support that
  2. I'm not sure it makes sense grammatically? "Against too much financial risk tolerance", "Against many arguments for financial risk tolerance"?

Starting with your conclusions:

Justifying a riskier portfolio

  • The “small fish in a big pond + idiosyncratic
... (read more)
5
trammell
8mo
Hi, sorry for the late reply--just got back from vacation. As with most long posts, I expect this post has whatever popularity it has not because many people read it all, but because they skimmed parts and thought they made sense, and thought the overall message resonated with their own intuitions. Likewise, I expect your comment has whatever popularity it has because they have different intuitions, and because it looks on a skim as though you’ve shown that a careful reading of the post validates those intuitions instead…! But who knows. Since there are hard-to-quantify considerations both for and against philanthropists being very financially risk tolerant, if your intuitions tend to put more weight on the considerations that point in the pro-risk-tolerance direction, you can certainly read the post and still conclude that a lot of risk tolerance is warranted. E.g. my intuition differs from yours at the top of this comment. As Michael Dickens notes, and as I say in the introduction, I think the post argues on balance against adopting as much financial risk tolerance as existing EA discourse tends to recommend. Beyond an intuition-based re-weighting of the considerations, though, you raise questions about the qualitative validity of some of the points I raise. And as long as your comment is, I think the post does already address essentially all these questions. (Indeed, addressing them in advance is largely why the post is as long as it is!) For example, regarding “arguments from uncertainty”, you say But to my mind, the way this flattening could work is explained in the “Arguments from uncertainty” section: Or, in response to my point that “The philanthropic utility function for any given “cause” could exhibit more or less curvature than a typical individual utility function”, you say Could you point me to what you're referring to, when you say you note this above? To my mind, one way that a within-cause philanthropic utility function could exhibit arbitraril

We have a much flatter utility curve for consumption (ignoring world-state) vs individual investors (using GiveWell's #s, or cause variety). [Strong]

Can you elaborate on why you believe this? Are you talking specifically about global poverty interventions, or (EA-relevant) philanthropy in general? (I can see the case for global poverty[1], I'm not so sure about other causes.)

We have a much lower correlation between our utility and risk asset returns. (Typically equities are correlated with developed market economies and not natural disasters) [Strong]

... (read more)
8[anonymous]9mo
I appreciate the way that this comment concretely engages with perceived issues in the post. I think this is great. I don't feel qualified to comment on the substance of the comment, but I do think that some of the style is needlessly hostile. Just in the first few lines, we have a nitpick about the grammar of the title, and this line which I think is both unpleasant and ultimately not very informative (compared to the more substantive critiques raised): "I'm honestly surprised [the post is] so popular." I think it would be possible to raise the substantive issues without making the forum a less friendly place. I also think that approaching things in this way would be practically more productive, in that I think hostility decreases the chance of honest reflection on issues raised. ETA: nevertheless, I did not downvote this comment because I think that such reflective and thoughtful engagement with a post should be encouraged.

Yes, I agree with this - editing the post to make this correction

Tyler Cowen on the effect of AGI on real rates:

In standard models, a big dose of AI boosts productivity, which in turn boosts the return on capital, which then raises real interest rates.

I am less convinced.  For one thing, I believe most of the gains from truly fundamental innovations are not captured by capital.  Was Gutenberg a billionaire?  The more fundamental the innovation, the more the import of the core idea can spread to many things and to many sectors.

Furthermore, over the centuries real rates of return seem to be falling, even th

... (read more)

There's a 3rd reason, which I expect is the biggest contributor. Number of readers of the post/comment.

2
Will Aldred
11mo
I think this 3rd explanation can be decomposed into the first two (and possibly others that I haven't laid out)? (For what it's worth, I agree with a "You don’t have to respond to every comment" norm. So, don't feel obligated to reply, especially if you think—as may well be the case—that I'm talking past you and/or that I'm missing something and the resulting inferential distance is large.)

I started writing a comment, but it got too long, so I wrote it up here.

I summarised a little bit how various organisations in the EA space aggregate QALY's over time here.

What I've been unable to find anywhere in the literature is how many QALYs a typical human life equates to? If I save a newborn from dying, is that worth 70 QALYs (~global life expectancy), 50 QALYs (not all of life is lived in good health), or some other value?

I think this post by Open Phil is probably related to what you're asking for and I would also recommend the GiveWell post on the same topic

I think this is still generally seen as a bit of an open ques... (read more)

How do you square:

The order was: I learned about one situation from a third party, then learned the situation described in TIME, then learned of another situation because I asked the woman on a hunch, then learned the last case from Owen.

with

No other women raised complaints about him to me, but I learned (in some cases from him) of a couple of other situations where his interactions with women in EA were questionable. 

Emphasis mine. (Highlighting your first statement implies he informed you of multiple cases and this statement implies he only informed you of one)

In the first case, I initially heard about the situation from a third party, but nearly all the information I knew came from Owen. (I asked the woman if she had concerns about the situation that she wanted to discuss, and I didn’t hear back.)

Please would someone be able to put together a slightly more fleshed out timeline of who knew what and when. Best I can tell is:

  • 3rd February 2023 - TIME article published
  • 3rd February 2023 - People start questioning this specific case in the forum
  • 3rd February 2023 - Julia and Owen discuss who should find out about this
  • 3rd February 2023 - Julia informed Nicole that the person was Owen
  • 3rdFebruary 2023 - Julia informs EV US and EV UK boards
  • 4th February 2023 - Julia informed Chana that the person was Owen
  • 11th February 2023 - Owen resigns from the board
  • 20th Febr
... (read more)

On Feb 3 I heard from Owen, I discussed the situation with Nicole, I informed Owen I'd be telling the boards, and I told the boards. I told Chana the following morning.

I know I'm probably being dense here, but would it be possible for you to share what the other possibilities are?

Edit: I guess there's "The person doesn't have the role, but we are bound by some kind of confidentiality we agreed when removing them from post"

6
ChanaMessinger
1y
No, it's a reasonable question. I hope to be able to answer these questions better next week. I'm really sorry, I know that's not very helpful.
3
Peter Wildeford
1y
I don't see how confidentiality would prevent anyone from literally saying "The person doesn't have the role, but we are bound by some kind of confidentiality we agreed when removing them from post", which would actually be a reassuring thing to hear.

Just bumping this in case you've forgotten. At the moment there only seem to be two possibities: 1/ you forgot about this comment or 2/ the person does still have a role "picking out promising students" as Peter asked. I'm currently assuming it's 2, and I imagine other people are too.

7
ChanaMessinger
1y
Hi Simon -  Two posts (from Owen and from the UK boards) + comments from me and Julia on the latter have just gone up that might have the updates you're looking for.

We are working actively on this, but it is going to take more time. As a general point (not trying to comment on this situation in particular), those are not the only two possibilities, and I think it's really crucial to be able to hold on to that in contexts where there's issues of legality, confidentiality and lots of imperfect information flow.

Edit note: I at first had "local point" instead of "general point", which I meant in a mathy way, like the local logic of the situation point rather than speaking to any of the context, but looking back I don't think that was very clear so I've edited to clarify my meaning.

iirc, there is access to the histogram, which tells you how many people predicted each %age. I then sampled k predictors from that distribution.

"k predictors" is the number of samples I was looking at
">N predictors" was the total number of people who predicted on a given question

what does SD stand for? Usually I would expect standard deviation? 

Yes, that's exactly right. The HLI methodology consists of polling together a bunch of different studies effect-sizes (measured in standard deviations) and then converting those standard deviations into WELLBYs. (By mulitplying by a number ~2). 

No bet from me on the Ozler tria

Fair enough - I'm open to betting on this with anyone* fwiw. * anyone who hasn't already seen results / involved in the trial ofc

Any intervention is extremely sensitive to implementation details, whether deworming or nets or psychotherapy.

Yes, I'm sorry if my comment appeared to dismiss this fact as I do strongly agree with this. 

Maybe some interventions are easier to implement than others,  and there might be  more variance  in the effectiveness of psychotherapy compared with net distribution (although I doubt that, I would guess less variance than nets)  but  all are very sensitive to implementation details.

This is pretty much my point

I'd be intereste

... (read more)
1
NickLaing
1y
Just to clarify, am not part of Strongminds or HLI, maybe you thought I was Joel replying? Thanks for the clarifications, appreciate that. Seems like we generally agree on implementation sensitivity. Thanks for your explanation on the HLI numbers which unfortunately I only I partly understand. A quick (and possibly stupid question), what does SD stand for? Usually I would expect standard deviation?  No bet from me on the Ozler trial I'm afraid (not a gambling guy ;) ). Personally I think this trial it will find a fairly large effect due partly to the intervention actually working, but the effect will be inflated compared to the real effect to a due to inflated post-study  SBJ scores. This happens due to "demand bias" and "future hope bias" (discussed in another post) but my certainty of any of this is so low it almost touches the floor...

My analysis of StrongMinds is based on a meta-analysis of 39 RCTS of group psychotherapy in low-income countries. I didn’t rely solely on StrongMinds’ own evidence alone, I incorporated the broader evidence base from other similar interventions too. This strikes me, in a Bayesian sense, as the sensible thing to do.

I agree, but as we have already discussed offline, I disagree with some of the steps in your meta-analyses, and think we should be using effect sizes smaller than the ones you have arrived at. I certainly didn't mean to claim in my post that... (read more)

5
NickLaing
1y
A couple of quick comments Simon. First on this comment, which I disagree with - and this is one of the few areas where I think the Effective Altruism community can at times miss something quite important. This isn't really about the StrongMinds charity question, but instead a general bugbear of mine as someone who implements things ;). " I think the difference here is I wouldn't expect those interventions to be as sensitive to the implementation details."  Any intervention is extremely sensitive to implementation details, whether deworming or nets or psychotherapy. In fact I think that intervention details are often more important than the pre-calculated expected value. If a given intervention is implemented poorly, or in the wrong place or at the wrong time then it could still have less impact than an intervention that is theoretically 100x worse. As a plausible if absurd example, imagine a vitamin A project which doesn't actually happen because the money is corrupted away.  Or if you you give out mosquito nets in the same village where another NGO gave out nets 2 weeks ago. Or if you deworm in a place where 20 previous deworming projects and sanitation has already drastically reduced the worm burden. Maybe some interventions are easier to implement than others,  and there might be  more variance  in the effectiveness of psychotherapy compared with net distribution (although I doubt that, I would guess less variance than nets)  but  all are very sensitive to implementation details.   And second this statement "just that it has an effect which is small enough that we are looking at numbers on the order (or lower) than cash-transfers and therefore it doesn't meet the bar of "Top-Charity"." I'd be interested in you backing up this comment with a bit explanation if you have time (all good if not!). I know this isn't your job and you don't have the time that Joel has, but  what is it that has led you to conclude that the numbers are "on the order (or lower) than

I had seen both of those, but I didn't read either of them as commitments that HLI thinks that the neutral point is between 0 and 5. 

I would guess some combination of:

  • Increasing longevity (which note the authors say has an effect in the FOOM scenario, but not in the aligned scenario...)
  • Decreasing credit risk (what was 'risk free' in the 1400s is very different to what is 'risk free' today)
  • Consumption preferences being correlated to growth

I don't really have a strong opinion on any of these - macro is really hard and really uncertain. To quote a friend of mine:

one thing is that if AGI looks something like robin hanson's EM scenario, you really don't want to owe money to anyone
or if other

... (read more)
4
Benjamin_Todd
1y
Thanks! Yeah I agree that the AGI could also make you want to save more. One factor is that higher interest rates can mean it's better to save more (depending on your risk aversion). Another factor is that it could increase your lifespan, or make it easier to convert money into utility (making your utility function more linear). That it could reduce the value of your future income from labour is another factor.

I'm really confused where any of those numbers have come from for using futures? (But yes, the expected return with low leverage is not spectacular for 2% move in rates).

3
Benjamin_Todd
1y
I think the key point is just equities will also go down if real interest rates rise (all else equal) and plausibly by more than a 20 year bond.

I want to suggest a bunch of caution against shorting bonds (or tips).

  • The 30yr yield is 3.5%, so you make -3.5% per year from that.
  • You earn the cash rate on the capital freed up from the shorts, which is 3.8% in interactive brokers.
  • If you're right that the real interest rate will rise 2% over 20 years, and average duration is 20 years, then you make +40% over 20 years – roughly 2% per year.
  • If you buy an ETF, maybe you lose 0.4% in fees.

So you end up with a +1.9% expected return per year.

I think the calculation you've done here is -3.5% + 3.8% + 2% - 0.4%

Th... (read more)

2
Benjamin_Todd
1y
Thanks that makes sense.   So if you implemented this with a future, you'd end up with -3.5% + 2.9%  + rerating return = -0.6% + rerating.  With a 2% p.a. re-rating return over 20 years, the expected return is +1.4%, minus any fees & trade management costs. If it happens over only 5 years, then +7.4%.

A 60:40 portfolio has an effective duration of ~40 years, where most of that duration comes from equities.

I'm not really sure how you get that? The duration on the bond portion is going to be ~7-10y which would imply 60y duration for equities, which I think is wrong.

My understanding is that an important part of the reasoning for a focus on avoiding bonds is that an increase in GDP growth driven by AI is clearly negative for bonds, but has an ambiguous effect on equities (plus commodities and real estate), so overall you should hold more equities (/growth a

... (read more)
2
Benjamin_Todd
1y
I think the effective duration on equities is roughly the inverse of the dividend yield + net buybacks, so with a ~2% yield, that's ~50 years. Some more here: https://www.hussmanfunds.com/wmc/wmc040223.htm

Yes - good point. I have fixed that

The highest neutral point we think is plausible is 5/10 on a 0 to 10 wellbeing scale, but we mentioned that some philosophical views would stake a claim to the feasibility of 10/10.

If you can point me to somewhere on the HLI website I can cite I will update this.

I think you can fill out the missing cells for HLI by taking the average age of death, which for Malaria I is ~2 for under 5's and ~46 for over 5s. Assuming a life expectancy of 70 (what we've assumed previously for malaria deaths), that'd imply a moral weight of under-5s = (70 - 2) * (-1 , 4) &nbs

... (read more)
2
JoelMcGuire
1y
Sure, See section 2.2 Or Or you could also note that we estimate the lower bound of the value of saving a life as assuming a neutral point of 5. 

This is one way of looking at the data (with my overlapping data claim already noted).

Here is another (much longer) way of looking at the data.

Here is 800 years of history courtesy of the BoE

Eyeballing that, I'd say the relationship is strongly negative...

3
Benjamin_Todd
1y
Interesting. What would be the theoretical explanation for a negative relationship?

Whenever I see charts like this in a financial context I twitch. We have 30 years of data for UK real rates, less for other issuers. There are ~2 non-overlapping UK data points on your second chart where I can count at least 15(?) data points?

From ishaan here.

To expand a little on "this seems implausible":  I feel like there is probably a mistake somewhere in the notion that anyone involves thinks that <doubling income as having 1.3 WELLBY and severe depression has having a 1.3 WELLBY effect.> 

 The mistake might be in your interpretation of HLI's document (it does look like the 1.3 figure is a small part of some more complicated calculation regarding the economic impacts of AMF and their effect on well being, rather than intended as a headline finding about the cash to well

... (read more)
1
ishaan
1y
My response to this post overall is that I think some of what is going on here is that different people and different organizations mean very different things when we say "Depression". Since "depression" is not really a binary, the value of averting "1 case of severe depression" can change  a lot depending on how you define severity, in such a way that differences in reasonable definitions of "sufficiently bad depression" can plausibly differ by 1-3x when you break it down into "how many SD counts as curing depression" terms. However, the in-progress nature of SoGives' mental health work makes pinning down what we do mean sort of tricky. What exactly did the participants in the SoGive Delphi Process mean when they said "severe depression"? How should I, as an analyst who isn't aiming to set the moral weights but is attempting to advise people using them, interpret that? These things are currently in flux, in the sense that I'm basically in the process of making various judgement calls about them right now, which I'll describe below. You commented: It's true that the PHQ-9 score of 27 points maxes out around 2-4sd. How many SD it is exactly depends on the spread of your population of course (for example if 1sd=6.1 points then the range of a 27 point scale spans 4.42sd ), and for some population spreads it would be 3sd.   These two things are related actually! I think the trouble is that the word "severity depression" is ambiguous as to how bad it is, so different people can mean different things by it.  One might argue that the following was an awkward workaround which should have been done differently, but basically, to make transparent my internal thought process here (In terms of what I thought after joining sogive, starting this analysis, and encountering these weights) was the following: -> "hm, this implies we're willing to trade averting 25 years of depression against one (mostly neonatal) death. Is this unusual?"  -> "Maybe we are thinking about the ty

that would be very important as it would mean that SoGive moral weights fail some basic sanity checks

I would recommend my post here. My opinion is - yes - SoGive's moral weights do fail a basic sanity check.

1 year of averted depression is 4 income doublings
1 additional year of life (using GW life-expectancies for over 5s) is 1.95 income doublings.

ie SoGive would thinks depression is worse than death. Maybe this isn't quite a "sanity check" but I doubt many people have that moral view.

I do think all this is a somewhat separate discussion from the GWWC list

I... (read more)

1
ishaan
1y
I replied in the moral weights post w.r.t. "worse than death" thing. (I think that's a fundamentally fair, but fundamentally different point from what I meant re: sanity checks w.r.t not crossing hard lower bounds w.r.t. the empirical effects of cash on well being vs the empirical effect of mental health interventions on well being)

I have a simple answer to this: no, it isn't.

I don't understand how that's possible. If you put 3x the weight on StrongMind's cost-effectiviness viz-a-vis other charities, changing this must move the needle on cost-effectiveness more than anything else. It's possible to me it could have been "well into the range of gold-standard" and now it's "just gold-standard" or "silver-standard". However if something is silver standard, I can't see any way in which your cost-effectivness being adjusted down by 1/3rd doesn't massively shift your rating.

I'd say that the

... (read more)
2
Sanjay
1y
Ishaan's work isn't finished yet, and he has not yet converted his findings into the SoGive framework, or applied the SoGive moral weights to the problem. (Note that we generally try to express our findings in terms of the SoGive framework and other frameworks, such as multiples of cash, so that our results are meaningful to multiple audiences). Just to reiterate, neither Ishaan nor I have made very strong statements about cost-effectiveness, because our work isn't finished yet. That sounds great, I'll message you directly. Definitely not wishing to misunderstand or misinterpret -- thank you for your engagement on this topic :-)

I agree - and I started out trying to list all their approaches, but it very quickly becomes untractable in the table format. I have edited to show the full range, although I'm not sure if it's more or less helpful than before. Hopefully it does should how counter-intuitive their model can be

2
JoelMcGuire
1y
Is this because we argued that it's plausible that a life can have negative wellbeing? 
5
MHR
1y
Thanks for the edit! I think that’s helpful

Please reply to this comment if there is another org you would like to see added to the grid.

I might be being a bit dim here (I don't have the time this week to do a good job of this), but I think of all the orgs evaluating StrongMinds, SoGive's moral weights are most likely to find favourably for StrongMinds. Given that, I wonder what you expect you'd rate them at if you altered your moral weights to be more inline with FP and HLI?

SoGive’s Gold Standard Benchmarks are:

  1. £5,000 per life saved
  2. £50 to double someone’s consumption (spending) for one year
  3. £200 to avert one year of severe depression
  4. £5 to avert the suffering of one chicken who is living in
... (read more)
4
ishaan
1y
To expand a little on "this seems implausible":  I feel like there is probably a mistake somewhere in the notion that anyone involves thinks that <doubling income as having 1.3 WELLBY and severe depression has having a 1.3 WELLBY effect.>   The mistake might be in your interpretation of HLI's document (it does look like the 1.3 figure is a small part of some more complicated calculation regarding the economic impacts of AMF and their effect on well being, rather than intended as a headline finding about the cash to well being conversion rate). Or it could be that HLI has an error or has inconsistencies between reports. Or it could be that it's not valid to apply that 1.3 number to "income doubling"  SoGive weights for some reason because it doesn't actually refer to the WELLBY value of doubling.   I'm not sure exactly where the mistake is, so it's quite possible that you're right, or that we are both missing something about how the math behind this works which causes this to work out, but I'm suspicious because it doesn't really fit together with various other pieces of information that I know. For instance -  it doesn't really square with how HLI reported Psychotherapy is 9x GiveDirectly when the cost of treating one person with therapy is around $80, or how they estimated that it took $1000 worth of cash transfers to produce 0.92 SDs-years of subjective-well-being improvement ("totally curing just one case of severe depression for a year" should correspond to something more like 2-5 SD-years).  I wish I could give you a clearer "ah, here is where i think the mistake is" or perhaps a "oh, you're right after all" but I too am finding the linked analysis a little hard to follow and am a bit short on time (ironically, because I'm trying to publish a different piece of Strongminds analysis before a deadline).  Maybe one of the things we can talk about once we schedule a call is how you calculated this and whether it works? Or maybe HLI will comment and clear things
3
ishaan
1y
Good stuff. I haven't spent that much time looking at HLIs moral weights work but I think the answer is "Something is wrong with how you've constructed weights, HLI is in fact weighing mental health harder than SoGive". I think a complete answer to this question requires me checking up on your calculations carefully, but I haven't done so yet, so it's possible that this is right. If if were true that HLI found anything on the order of roughly doubling someone's consumption  improved well being as much as averting 1 case of depression, that would be very important as it would mean that SoGive moral weights fail some basic sanity checks. It would imply that we should raise our moral weight on cash-doubling to at least match the cost of therapy even under a purely subjective-well-being oriented framework to weighting. (why not pay 200 to double income, if it's as good as averting depression and you would pay 200 to avert depression?) This seems implausible. I haven't actually been directly researching the comparative moral weights aspect, personally - I've been focusing primarily on <what's the impact of therapy on depression in terms of effect size> rather than on the "what should the moral weights be" question (though I have put some attention to the "how to translate effect sizes into subjective intuitions" question, but that's not quite the same thing). That said when I have more time I will look more deeply into this and check if our moral weights are failing some sort of sanity check on this order, but, I don't think that they are. Regarding the more general question of "where would we stand if we altered our moral weights to be something else", ask me again in a month or so when all the spreadsheets are finalized, moral weights should be relatively easy to adjust once the analysis is done.  (as sanjay alludes to in the other thread, I do think all this is a somewhat separate discussion from the GWWC list - my main point with the GWWC list was that StrongMind
8
Sanjay
1y
Thanks for your question Simon, and it was very eagle-eyed of you to notice the difference in moral weights. Good sleuthing! (and more generally, thank you for provoking a very valuable discussion about StrongMinds) I run SoGive and oversaw the work (then led by Alex Lawsen) to produce our moral weights. I'd be happy to provide further comment on our moral weights, however that might not be the most helpful thing. Here's my interpretation of (the essence of) your very reasonable question: I have a simple answer to this: no, it isn't. Let me flesh that out. We have (at least) two sources of information: 1. Academic literature 2. Data from StrongMinds (e.g. their own evaluation report on themselves, or their regular reporting) And we have (at least) two things we might ask about:          (a) How effective is the intervention that StrongMinds does, including the quality of evidence for it?          (b) How effective is the management team at StrongMinds? I'd say that the main crux is the fact that our assessment of the quality of evidence for the intervention (item (a)) is based mostly on item 1 (the academic literature) and not on item 2 (data from StrongMinds). This is the driver of the comments made by Ishaan above, not the moral weights. And just to avoid any misunderstandings, I have not here said that the evidence base from the academic literature is really robust -- we haven't finished our assessment yet. I am saying that (unless our remaining work throws up some surprises) it will warrant a more positive tone than your post, and that it may well demonstrate a strong enough evidence base + good enough cost-effectiveness that it's in the same ballpark as other charities in the GWWC list.

Out of interest what do your probabilities correspond to in terms of the outcome from the Ozler RCT? (Or is your uncertainty more in terms of what you might find when re-evaluating the entire framwork?)

3
Matt_Lerner
1y
I haven't thought extensively  about what kind of effect size I'd expect, but I think I'm roughly 65-70% confident that the RCT will return evidence of a detectable effect. But my uncertainty is more in terms of rating upon re-evaluating the whole thing. Since I reviewed SM last year, we've started to be a lot more punctilious about incorporating various discounts and forecasts into CEAs. So on the one hand I'd naturally expect us to apply more of those discounts on reviewing this case, but on the other hand my original reason for not discounting HLI's effect size estimates was my sense that their meta-analytic weightings appropriately accounted for a lot of the concerns that we'd discount for. This generates uncertainty that I expect we can resolve once we dig in.

since we base our assessment mostly on HLI's work, and since we draw different conclusions from HLI's work than you think are reasonable, we should reassess StrongMinds on that basis. Is that right

I'm not sure exactly what you've done, so it's hard for me to comment precisely. I'm just struggling to see how you can be confident in a "6x as effective as GD" conclusion.

what does a distribution of your beliefs over cost-effectiveness for StrongMinds look like?

So there are two sides to this:

  1. Is my confidence in HLI's philisophical views. I have both spoken to J
... (read more)

I agree that beforemy post GWWC hadn't done anything wrong.

At this point I think that GWWC should be able to see that their current process for labelling top-rated charities is not optimal and they should be changing it. Once they do that I would fully expect that label to disappear.

I'm disappointed that they don't seem to agree with me, and seem to think that no immediate action is required. Obviously that says more about my powers of persuasion than them though, and I expect once they get back to work tomorrow and they actually look in more detail they c... (read more)

Hi Simon,

I'm back to work and able to reply with a bit more detail now (though also time-constrained as we have a lot of other important work to do this new year :)).

I still do not think any (immediate) action on our part is required. Let me lay out the reasons why:

(1) Our full process and criteria are explained here. As you seem to agree with from your comment above we need clear and simple rules for what is and what isn't included (incl. because we have a very small team and need to prioritize). Currently a very brief summary of these rules/the process... (read more)

As a GWWC member who often donates through the GWWC platform I think it is great that they take a very broad brush and have lots of charities that people might see as top on the platform. I think if their list got to small they would not be able to usefully serve the GWWC donor community (or other donors) as well.

I agree, and I'm not advocating removing StrongMinds from the platform, just removing the label "Top-rated". Some examples of charities on the platform which are not top-rated include: GiveDirectly, SCI, Deworm the World, Happier Lives Institute, ... (read more)

4
weeatquince
1y
Ah. Good point. Replied to the other thread here: https://forum.effectivealtruism.org/posts/ffmbLCzJctLac3rDu/strongminds-should-not-be-a-top-rated-charity-yet?commentId=TMbymn5Cyqdpv5diQ .
Load more