Terrific overview! I'll offer some feedback with the hope that some of it may be helpful:

Big Picture Thoughts

1. In general, I thought the report did a great job summarizing some of the major themes/ideas that are fairly well-established in global mental health. I wonder if it could be useful to include a section on more experimental/novel/unestablished/speculative ideas. Sort of like a "higher risk, higher potential reward" section.
2. Relatedly, I'd be interested in seeing bolder and more specific recommendations for future work. As an example, Box 2 ("Promising Research Directions") lists important goals, but they're too broad to really know how to act on (e.g., "improve treatments and expand access to care."). I'd be more curious to see HLI's subjective opinions on the most impactful next steps (more similar to the list of project ideas that you have, rather than the goals in Box 2).
3. I'd love to see more analysis on key issues/controversies (see last section for examples).

Potentially useful points that I didn't see in the report:

1. A lot of suffering is caused by subclinical/subsyndromal mental health problems. In the case of mood disorders, "subsyndromal symptoms are impairing, predict syndrome
   onset and relapse, and account for more doctor’s visits and suicide attempts
   than the full syndromes." (Ruscio, 2019). This point is especially important because there are debates about how funding should be allocated (e.g., how much should we spending on treatments that target people with diagnosable disorders vs. mental health promotion strategies and prevention programs that reach broader audiences?)
2. Recent work has suggested that the "latent disease" view of depression (and other mental disorders) may be flawed (e.g., Borsboom, 2017). A related body of work has suggested that some depressive symptoms may be more impairing than others (e.g., Fried & Nesse, 2014). This could have important implications for measuring the effectiveness of interventions-- e.g., estimating SWB weights for each symptom, rather than using sum-scores.
3. The evidence on task-sharing/task-shifting is strong, so I understand why you spent a lot of space covering it. At the same time, it could be useful to spend more time discussing some of the more novel approaches. Some examples include unguided self-help interventions and single-session interventions (Schleider & Weisz, 2017). Although the evidence for guided interventions and longer interventions is stronger, unguided interventions are substantially cheaper. This might make them more cost-effective, even if longer/guided interventions are more effective (discussed further in this preprint).
4.  The digital interventions studied in meta-analyses and reviews are very different than those that have been disseminated widely. We know a lot about the effectiveness of digital interventions developed by professors, but much less about the effectiveness of Headspace, Calm, and other popular apps (Wasil et al., 2019).
5.  There's are some important gaps in the digital mental health space: popular interventions tend to focus on relaxation/mindfulness and rarely include other empirically supported treatment elements (Wasil et al., 2020). This reminds me that I really should write up a digital mental health forum post at some point :) 

Examples of questions/controversies that HLI could address:

1. Broadly, what does HLI see as some of the most important open questions in the mental health space? 
2. What content should be included in interventions? Does HLI believe that specific elements should be the focus of interventions? Or are common factors driving effects?
3. Which delivery formats be used? Is HLI optimistic or pessimistic about unguided self-help interventions? Are they likely to be more cost-effective than task-sharing interventions?
4. Does HLI see mental disorders as diseases, networks of symptoms, or something else? Do you think this matters, or not really?
5. Broadly, what does HLI think that a lot of people interested in mental health "get wrong" or "don't yet know" about the most cost-effective ways to make an impact?
6. How long do the effects of interventions last? How should the uncertainty around this estimate affect our cost-effectiveness calculations? (assuming that the effects of an intervention will last <1 year seems like it would yield radically different conclusions than assuming it would 1-3 years, 3+ years, 10+ years 30+ years, etc.)

I hope that some of this was helpful & I'm looking forward to seeing future reports!

I think the steelman of the neglectedness argument would be something like: "The less neglected something is, the less likely it is that we would be able to make them do it slightly better."

This is both because (a) it is harder to change the direction of the movement and (b) it is harder to genuinely find meaningful ways to improve the movement.

In (b), I wonder if there are some specific limitations of the current War-on-Drugs movement that would match the skills/interests of (some) EAs. 

I'd be curious to learn more about the "types" of EAs that might be best-suited for this work, or how the "EA perspective" could enhance ongoing efforts.

As it stands, the case for scale (i.e., the magnitude of the problem) is very clear. However, I think scale is usually the strongest part of most cause area analyses (i.e., there are a lot of really big problems and it's usually not too difficult to articulate the bigness of those problems, especially using words rather than models). I think the role that EAs would play is less clear (as has been reflected in other comments relating to neglectedness). So, I wonder:

Are there some clear gaps or limitations in the current anti-War-on-drugs movement that could be filled by EA perspectives/skills? (As an example, one of the commentators emphasized that global efforts to legalize drugs may be neglected, and EAs who have skills/interests related to global advocacy might be especially helpful).

What a great opportunity! I wonder if people at SparkWave (e.g., Spencer Greenberg), Effective Thesis, or the Happier Lives Institute would have some ideas. All three organizations are aligned with EA and seem to be in the business of improving/applying/conducting social science research.

Also, I have no idea who your advisor is, but I think a lot of advisors would be open to having this kind of conversation (i.e., "Hey, there's this funding opportunity. We're not eligible for it, but I'm wondering if you have any advice..."). [Context: I'm a PhD student in psychology at UPenn.]

If that's not a good option, you could consider asking your advisor (and other academics you respect) if they know about any metascience/open science organizations that are highly effective [without mentioning anything about your relative and their interest in donating].

Finally, it's not clear to me if the donor is only interested in metascience or if they would also be open to funding "basic science" projects. "Basic science" is broad enough that I imagine it could open up a lot of alternative paths (many of which might be more explicitly EA-aligned than metascience). Examples include basic scientific research on effective giving, animal advocacy, mental health, AI safety, etc. Do you have a sense of how open to "basic science" your relative is, or was basic science just meant as a synonym for metascience?

Finally, good luck on this! :)

Super exciting work! Sharing a few quick thoughts:

1. I wonder if you've explored some of the reasons for effect size heterogeneity in ways that go beyond formal moderator analyses. In other words, I'd be curious if you have a "rough sense" of why some programs seem to be so much better than others. Is it just random chance? Study design factors? Or could it be that some CT programs are implemented much better than others, and there is a "real" difference between the best CT programs and the average CT programs?

This seems important because, in practice, donors are rarely deciding between funding the "average" CT program or the "average" [something else] program. Instead, they'd ideally want to choose between the "best" CT program to the "best" [something else] program. In other words, when I go to GiveWell, I don't want to know about the "average" Malaria program or the "average" CT program-- I want to know the best program for each category & how they compare to each other.

This might become even more important in analyses of other kinds of interventions, where the implementation factors might matter more. For instance, in the psychotherapy literature, I know a lot of people are cautious about making too many generalizations based on "average" effect sizes (which can be weighed down by studies that had poor training procedures, recruited populations that were unlikely to benefit, etc.). 

With this in mind, what do you think is currently the "best" CT program, and how effective is it?

2. I'd be interested in seeing the measures that the studies used to measure life satisfaction, depression, and subjective well-being. 

I'm especially interested in the measurement of life satisfaction. My impression is that the most commonly used life satisfaction measure (this one) might lead to an overestimation of the relationship between CTs and life satisfaction. I think two (of the five) the items could prime people to think more about their material conditions than their "happiness." Items listed below:

• The conditions of my life are excellent (when people think about "conditions," I think many people might think about material/economic conditions moreso than affective/emotional conditions).
• So far I have gotten the important things I want in life (when people think about  things they want, I think many people will consider material/economic things moreso than affective/emotional things)

I have no data to suggest that this is true, so I'm very open to being wrong. Maybe these don't prime people toward thinking in material/economic terms at all. But if they do, I think they could inflate the effect size of CT programs on life satisfaction (relative to the effect size that would be found if we used a measure of life satisfaction that was less likely to prime people to think materialistically).

Also, a few minor things I noticed:

1. "The average effect size (Cohen’s d) of 38 CT studies on our composite outcome of MH and SWB is 0.10 standard deviations (SDs) (95% CI: 0.8, 0.13)."

I believe there might be a typo here-- was it supposed to be "0.08, 0.13"?

2. I believe there are two "Figure 5"s-- the forest plot should probably be Figure 6. 

Best of luck with next steps-- looking forward to seeing analyses of other kinds of interventions!

What are the things you look for when hiring? What are some skills/experiences that you wish more EA applicants had? What separates the "top 5-10%" of EA applicants from the median applicant?

Thank you, Denise! I think this gives me a much better sense of some specific parts of the post that may be problematic.  I still don't think this post, on balance, is particularly "bad" discourse (my judgment might be too affected by what I see on other online discussion platforms-- and maybe as I spend more time on the EA forum, I'll raise my standards!). Nonetheless, your comment helped me see where you're coming from.

I'll add that I appreciated that you explained why you downvoted, and it seems like a good norm to me. I think some of the downvotes might just be people who disagree with you. However, I also think some people may be reacting to the way you articulated your explanation. I'll explain what I mean below:

In the first comment, it seemed to me (and others) like you assumed Mark intentionally violated the norms. You also accused him of being unkind and uncurious without offering additional details. 

In the second comment, you linked to the guidelines, but you didn't engage with Mark's claim ("I think this was kind and curious given the context."). This seemed a bit dismissive to me (akin to when people assume that a genuine disagreement is simply due to a lack of information/education on the part of the person they disagree with).

In the third comment (which I upvoted), you explained some specific parts of the post that you found excessively unkind/uncivil. This was the first comment where I started to understand why you downvoted this post.

To me, this might explain why your most recent post has received a lot of upvotes. In terms of "what to make of this," I hope you don't conclude "users should not explain why they downvote." Rather, I wonder if a conclusion like "users should explain why they downvote comments, and they should do so in ways that are kind & curious, ideally supported by specific examples when possible" would be accurate. Of course, the higher the bar to justify a downvote, the fewer people will do it, and I don't think we should always expect downvote-explainers to write up a thorough essay on why they're downvoting. 

Finally, I'll briefly add that upvotes/downvotes are useful metrics, but I wouldn't place too much value in them. I'm guessing that upvotes/downvotes often correspond to "do I agree with this?" rather than "do I think this is a valuable contribution?"  Even if your most recent comment had 99 downvotes, I would still find it helpful and appreciate it!

Thank you for this post, Mark! I appreciate that you included the graph, though I'm not sure how to interpret it. Do you mind explaining what the "recommendation impression advantage" is? (I'm sure you explain this in great detail in your paper, so feel free to ignore me or say "go read the paper" :D).

The main question that pops out for me is "advantage relative to what?" I imagine a lot of people would say "even if YouTube's algorithm is less likely to recommend [conspiracy videos/propaganda/fake news] than [traditional media/videos about cats],  then it's still a problem! Any amount of recommending [bad stuff that is  harmful/dangerous/inaccurate] should not be tolerated!"

What would you say to those people?

I read this post before I encountered this comment. I didn't recall seeing anything unkind or uncivil. I then re-read the post to see if I missed anything.

I still haven't been able to find anything problematic. In fact, I notice a few things that I really appreciate from Mark. Some of these include:

• Acknowledging explicitly that he's sometimes rude to his opponents (and explaining why)
• Acknowledging certain successes of those he disagrees with (e.g., "I'll give this win to Tristan and Roose.")
• Citing specific actions/quotes when criticizing others (e.g., the quote from the Joe Rogan podcast)
• Acknowledging criticisms of his own work 

Overall, I found the piece to be thoughtfully written & in alignment with the community guidelines. I'm also relatively new to the forum, though, so please point out if I'm misinterpreting the guidelines.

I'll also add that I appreciate/support the guideline of "approaching disagreements with curiosity" and "aim to explain, not persuade." But I also think that it would be a mistake to overapply these. In some contexts, it makes sense for a writer to "aim to persuade" and approach a disagreement from the standpoint of expertise rather than curiosity. 

Like any post, I'm sure this post could have been written in a way that was more kind/curious/community-normsy. But I'm struggling to see any areas in which this post falls short. I also think "over-correcting" could have harms (e.g., causing people to worry excessively about how to phrase things, deterring people from posting, reducing the clarity of posts, making writers feel like they have to pretend to be super curious when they're actually trying to persuade).

Denise, do you mind pointing out some parts of the post that violate the writing guidelines? (It's not your responsibility, of course, and I fully understand if you don't have time to articulate it. If you do, though, I think I'd find it helpful & it might help me understand the guidelines better.)

Thank you, Michael! I think this hypothetical is useful & makes the topic easier to discuss.

Short question: What do you mean by "user error?" 

Longer version of the question:

Let's assume that I fill out weights for the various categories of desire (e.g., health, wealth, relationships) & my satisfaction in each of those areas.

Then, let's say you erase that experience from my mind, and then you ask me to rate my global life satisfaction.

Let's now assume there was a modest difference between the two ratings. It is not instinctively clear to me why I should prefer judgment #1 to judgment #2. That is, I think it's an open question whether the "desire-based life satisfaction judgment" or the "desire-free life satisfaction judgment" is the more "valid" response.

To me, "user error" could mean several things:

• The "desire-free" judgment is flawed because the user is not thinking holistically enough or reflecting enough. They are not thinking carefully about what they care about & how those things have actually went. 
• The "desire-based" judgment is flawed because the list of desires misses some things that the user actually finds important (i.e., it's impossible to create a comprehensive list)
• The "desire-based" judgment is flawed because the user is not assigning weights properly (i.e., I might report that wealth matters twice as much to my life satisfaction than friendship, but I might be misperceiving my true preferences, which are better reflected in the "desire-free" case).

In other words, if we could eliminate these forms of user error, I would probably agree with you that this distinction is arbitrary. In practice, though, I think these "desire-based" and "desire-free" versions of life satisfaction ought to be considered distinct (albeit I'd expect them to be modestly correlated). I also don't think it's clear to me that the "desire-based" judgment should be considered better (i.e., more valid). And even if it should be considered better, I think I'd still want to know about the

Furthermore, when making decisions, I would probably want to see both judgments. For example, let's assume:

• Intervention A improves "desire-based life satisfaction judgments" by 15% and "desire-free life satisfaction judgments" by 5%
• Intervention B improves "desire-based life satisfaction judgments" by 10% and "desire-free life satisfaction judgments" by 10%
• Intervention C improves "desire-based life satisfaction judgments" by 15% and "desire-free life satisfaction judgments" by 15%.

I would prefer Intervention C over intervention A, even though they both improve "desire-based satisfaction judgments" by the same amount.  I also think reasonable people would disagree when comparing Intervention A to Intervention B.

For these reasons, I wonder if it's practically useful to consider "desire-based" and "desire-free" life satisfactions as separate constructs.