tl;dr – Comment below with arguments for why psychedelic research & advocacy should not be an EA cause area. On June 3rd, the most upvoted argument will win $400. The second- and third-most upvoted arguments will win $200 & $100, respectively.
1. Background on why I’m making this post & offering a prize
This effort was inspired by Grue_Slinky’s prize for the best argument against the EA Hotel.
I’m offering a $400 USD prize for the best argument against psychedelic research & advocacy being an EA cause area. I’m also offering $200 for the second-best argument & $100 for the third-best.
Uncovering another high-impact cause area seems very valuable, as EA’s impact is closely correlated with the quantity & quality of the cause areas it supports. This prize is a small effort towards assessing whether psychedelics should be taken up as an EA cause area.
The primary motivation here is to surface previously unconsidered arguments for why psychedelics should not be an EA cause area. My current view is that psychedelics are an extremely promising altruistic cause area – I’d like to stress-test that view to see what I’ve missed.
Below, I offer some arguments for why psychedelics should be an EA cause area.[1] These arguments are provided as a jumping-off point for arguments against.
2. Specification of the prize
To be considered for the prize, add your argument against psychedelics being an EA cause area as a top-level comment on this post. (Only top-level comments on this post will be considered as valid submissions.)
On Monday, June 3rd, I’ll award $400 USD to the most-upvoted argument against, $200 USD to the second-most-upvoted argument against, and $100 USD to the third-most-upvoted argument against. I will break any ties, by choosing my favorite of the tied arguments.
I’ll contact the prize winners via a Forum direct message to coordinate the remittance of their prizes. Once payouts are complete, I’ll announce the winners in a comment on this post.
Feel free to upvote other people’s submissions, even if you submit an argument yourself. (I suppose you could also downvote other submissions, though that doesn’t feel very sporting.)
To avoid potential bias, I will not upvote or downvote any prize submissions.
You can also comment with things that aren’t arguments against psychedelics being an EA cause area, though note that such comments will not be considered as prize submissions.
3. Arguments for the importance of psychedelics
To seed discussion, here are some arguments for the importance of psychedelics as an altruistic cause area.
3(a). Psychedelics are appealing to both long-termist & short-termist views
Almost everyone in EA holds either a long-termist view (everyone who will exist over the entire of course of the future deserves moral consideration) or a short-termist view (only people who exist before some time horizon deserve moral consideration).
If you hold a long-termist view (i.e. all future people deserve moral consideration) & you're a consequentialist (as far as I know, almost every long-termist is consequentialist), then it’s very difficult to do long-term cause prioritization, because you have limited visibility into the outcomes occurring 10,000+ years into the future – and these are outcomes you want to know about when assessing today’s actions. Practically, we're clueless about many long-term consequences of our present actions. You can read more about consequentialist cluelessness here.
Given this theoretical problem, long-termist cause prioritization should include “robustness to cluelessness” as a major factor.
Some x-risk interventions seem pretty robust to cluelessness, e.g. Eliezer Yudkowsky’s work raising awareness of the AI alignment problem. Eliezer’s advocacy work seems robust because in almost every scenario you can imagine, it’s good for more researchers to be aware of the AI alignment issue.[2]
Interventions that increase the set of well-intentioned + capable people also seem quite robust to cluelessness, because they allow for more error correction at each timestep on the way to the far future.
Rationality training programs like CFAR & Paradigm Academy are aimed at increasing the number of well-intentioned + capable people.
The psychedelic experience also seems like a plausible lever on increasing capability (via reducing negative self-talk & other mental blocks) and improving intentions (via ego dissolution changing one's metaphysical assumptions).
I compare the mechanisms of impact of the psychedelic experience and of programs like CFAR & Paradigm below.
By “changing one’s metaphysical assumptions,” I mean that the psychedelic state can change views about what the self is, and what actions constitute acting in one’s “self-interest.” Consider Michael Pollan’s account of one of his psilocybin trips, in How to Change Your Mind:
“I” now turned into a sheaf of little papers, no bigger than Post-its, and they were being scattered to the wind. But the “I” taking in this seeming catastrophe had no desire to chase after the slips and pile my old self back together. No desires of any kind, in fact. Whoever I now was was fine with whatever happened. No more ego? That was okay, in fact the most natural thing in the world.
…
For what was observing the scene was a vantage and mode of awareness entirely distinct from my accustomed self; in fact I hesitate to use the “I” to denote the presiding awareness, it was so different from my usual first person.
Where that self had always been a subject encapsulated in this body, this one seemed unbounded by any body, even though I now had access to its perspective… Everything I once was and called me, this self six decades in the making, had been liquefied and dispersed over the scene. What had always been a thinking, feeling, perceiving subject based in here was now an object out there.
Like Pollan describes, people often experience extremely altered self-awareness during psychedelic trips. This is part of the “mystical experience,” which appears to be correlated with the therapeutic benefits of psychedelics (see Roseman et al. 2017, Griffiths et al. 2008).
To sum up, under a long-termist view, psychedelic interventions are plausibly in the same ballpark of effectiveness of other interventions that increase the set of well-intentioned + capable people. This is because these interventions seem quite robust to consequentialist cluelessness – the extreme difficulty of confidently assessing far-future consequences of today's actions.
If you hold a short-termist view (i.e. only people who exist before some time horizon deserve moral consideration), mental health appears to be a cause area on par with global poverty, i.e. if global poverty interventions meet the criteria for being an EA cause area, so too should mental health interventions.
Mental illness appears to cause more suffering than poverty in developed countries, and it seems to cause roughly as much suffering worldwide as poverty does. Unlike poverty, the mental health burden isn’t shrinking. (See Michael Plant's cause profile on mental health for an analysis of how mental illness causes roughly as much suffering as global poverty.)
Psychedelics are showing a ton of promise as treatment for a battery of chronic mental health issues: anxiety (see Gasser et al. 2015, Griffiths et al. 2016), depression (see Carhart-Harris et al. 2018, Palhano-Fontes et al. 2019), OCD (see Moreno et al. 2006), and addictive disorders including smoking (see Johnson et al. 2017) & alcoholism (see Bogenschutz et al. 2015, Krebs & Johansen 2012).
For a summary of some of these findings, see dos Santos et al. 2016, a systematic review of the mental health effects of psychedelic therapy.
So, under a short-termist view, psychedelic interventions are plausibly in the same ballpark of effectiveness as global poverty interventions.
In summary, psychedelic research & advocacy seems particularly compelling as a cause area, because it performs well under both short-termist and long-termist worldviews. Given our uncertainty about morality, this appears to make psychedelics more robust than cause areas that rely on a single sequence of reasoning to justify their impact.
3(b). Comparison of rationality training to the psychedelic experience
As above, both rationality training and the psychedelic experience seem to be levers that increase the number of well-intentioned + capable people. This is plausibly robust to cluelessness (the difficulty of assessing outcomes that occur 10,000s of year in the future), because well-intentioned + capable people can course correct as we head into the future.
Comparing rationality training programs (e.g. CFAR, Paradigm Academy) to psychedelic experiences is tricky. It's hard to make an apples-to-apples comparison, because the interventions are operating on very different levels of abstraction.
The rationality training programs I know of operate almost entirely on the conceptual level (though I believe Paradigm uses some bodywork modalities also). The basic structure of conceptual rationality training is something like:
- Instructor says some words about a rationality topic
- Trainee hears these words & tries to internalize the topic
- Trainee practices their internalized version of the rationality topic (by themselves, with other trainees, or with the instructor)
- Instructor provides feedback to trainee to improve the trainee's internalized model of the topic
I think this structure can work really well for information & technique transfer, especially when the trainee is engaged & the instructor is skillful.
The basic structure of a psychedelic trip is very different:
- Participant thinks about and articulates the intentions & expectations they have for their upcoming psychedelic experience (to themselves, or to a facilitator)
- Participant ingests a psychedelic (by themselves, or with a sober facilitator present)
- Participant has a psychedelic experience. This experience can include a wide range of subjective elements:
- Old memories can come up and/or become salient
- New perspectives on relationships with friends, family, one's immediate environment can be adopted
- Emotions can be felt very intensely, especially emotions about salient people & topics in the one’s life
- Insights can be had about the participant's psychology, social assumptions, epistemic assumptions, and metaphysical assumptions
- New personal narratives ("this is the story of my life; this is what my life's about") can be adopted
- Once sober, participant integrates the experience (by themselves, or in dialogue a facilitator)
- How did the actual trip match up to your expectations about the trip?
- What came up? What was interesting? What was trivial, or silly?
- Did anything come up that's worth incorporating into your everyday life?
(See the Psychedelic Explorer’s Guide for more on how to structure an effective, safe trip.)
I think this structure can be really helpful for surfacing emotional blocks (e.g. things that generate akrasia, i.e. acting against one’s better judgment), as well as for resolving known emotional blocks.
The psychedelic experience can also help change one's assumptions, internal monologue, and personal narrative. (See the above Michael Pollan quote for an example of this. Also note that the psychedelic experience doesn't do this automatically, it can just help "loosen you up." The participant still has to intentionally work towards changing these things.)
So, to the extent that the EA community is limited by information & technique transfer, I'd expect conceptual rationality training to be more leveraged.
To the extent that the EA community is limited by emotional blocks & unhelpful personal narratives, I'd expect the psychedelic experience to be more leveraged.
My current view is that the EA community is more limited by emotional blocks & unhelpful personal narratives. The 2019 Slate Star Codex reader survey offers some data here: 17.4% of survey respondents have a formal diagnosis of depression (another 16.7% suspect they are depressed but haven’t been diagnosed); 12.6% of respondents have a formal diagnosis of anxiety (another 18.7% suspect they have anxiety but haven’t been diagnosed).
3(c). Interlude – on quantitative comparison
The above is merely a theoretical comparison – a quantitative comparative analysis would be helpful for thinking through the potential of psychedelics relative to other causes. Here’s one attempt at quantifying the benefits of a psychedelic intervention using DALYs, concluding that the intervention has a cost-per-DALY-averted of $472.
Note that DALY & QALY frameworks probably underweight mental health interventions due to the retrospective survey methodology used to construct their weightings.
Also note that there aren't many quantitative analyses that compare across EA cause areas. (e.g. comparing animal welfare interventions to global poverty interventions, or comparing x-risk interventions to animal welfare interventions.) Michael Dickens' cause prioritization app attempts this, though as far as I know it hasn't been used to drive much decision-making.
3(d). Trauma alleviation
Childhood trauma is plausibly upstream of several burdensome problems. See this excerpt from The Body Keeps Score, a pop-sci review of academic trauma research (on p. 150):
The first time I heard Robert Anda present the results of the ACE study, he could not hold back his tears. In his career at the CDC he had previously worked in several major risk areas, including tobacco research and cardiovascular health.
But when the ACE study data started to appear on his computer screen, he realized that they had stumbled upon the gravest and most costly public health issue in the United States: child abuse.
[Anda] had calculated that its overall costs exceeded those of cancer or heart disease and that eradicating child abuse in America would reduce the overall rate of depression by more than half, alcoholism by two-thirds, and suicide, IV drug use, and domestic violence by three-quarters. It would also have a dramatic effect on workplace performance and vastly decrease the need for incarceration.
Psychedelic therapy seems very promising for resolving PTSD, which could plausibly break the cycle of abuse that creates new traumatic experiences. (Trauma appears to transfer from generation to generation via multiple pathways.)
In particular, MDMA-assisted psychotherapy for PTSD is yielding extremely promising results in recent randomized controlled trials (see Mithoefer et al. 2012, Mithoefer et al. 2018, Ot’alora et al. 2018). From the abstract of Mithoefer et al. 2018:
At the primary endpoint, the 75 mg and 125 mg groups had significantly greater decreases in PTSD symptom severity (mean change CAPS-IV total scores of −58·3 [SD 9·8] and −44·3 [28·7]; p=0·001) than the 30 mg group (−11·4 [12·7]). Compared with the 30 mg group, Cohen's d effect sizes were large: 2·8 (95% CI 1·19–4·39) for the 75 mg group and 1·1 (0·04–2·08) for the 125 mg group.
…
PTSD symptoms were significantly reduced at the 12-month follow-up compared with baseline after all groups had full-dose MDMA (mean CAPS-IV total score of 38·8 [SD 28·1] vs 87·1 [16·1]; p<0·0001).
A Cohen’s d of 2.8 is extremely large (“Cohen suggested that d = 0.2 be considered a 'small' effect size, 0.5 represents a 'medium' effect size and 0.8 a 'large' effect size” source). Here’s a good resource for interpreting Cohen’s d.
In this study, 30 mg of MDMA was used as an active placebo, and the intervention groups were given 75 mg or 125 mg of MDMA.
From Mithoefer et al. 2012, a long-term follow-up of the first MDMA RCT:
We found the majority of these subjects with previously severe PTSD who were unresponsive to existing treatments had symptomatic relief provided by MDMA-assisted psychotherapy that persisted over time...
MDMA helped resolve severe PTSD symptoms in patients who had not responded to other treatment regimens. For 86% of patients, this benefit persisted 17+ months after the MDMA session.
3(e). Other (speculative) arguments for impact
Below are a few other mechanisms by which psychedelics could be highly impactful. These are speculative and I suggest holding them weakly – most of my own interest in psychedelic research & advocacy as an EA cause area is driven by the arguments given above.
Psychedelics may boost creativity & problem-solving. See Harman et al. 1966 – 27 white-collar professionals took mescaline and worked on a problem they had been stuck on.[3]
Outcomes for the problems participants had been stuck on:
- A commercial building design, accepted by the client
- Design of a linear electron accelerator beam-steering device
- Engineering improvement to a magnetic tape recorder
- A chair design, modeled and accepted by the manufacturer
- A mathematical theorem regarding NOR gate circuits
- Design of a private dwelling, approved by the client
Anecdotally, Steve Jobs said that taking LSD was “one of the most important things in my life.”
Eric Weinstein (mathematician, venture capitalist), Sam Harris (public intellectual), Aldous Huxley (writer), Kary Mullis (inventor of the polymerase chain reaction), and Douglas Engelbart (computer scientist, inventor of the computer mouse) have all spoken highly of psychedelics and/or are known to use them.
Psychedelics may expand one’s circle of moral concern. As far as I know, there hasn’t yet been a randomized study of the relationship between psychedelic use & altruism. (Though I’d love to see one!)
Lerner & Lyvers 2006 found that psychedelic users scored higher on empathy scales than non-drug users, though non-psychedelic drug users also had higher empathy scores than non-drug users, so it’s not clear if there was an effect driven by psychedelic use.
Anecdotally, see this essay by anonymous philanthropist Pine, who attributes their decision to set up the Pineapple Fund to a psychedelic experience: “I’ve had the idea to donate a large portion of my bitcoins for a while, and it is through this journey of discovering myself [with the help of psychedelics] that made me actually commit to it.”
Info hazard. Finally, there’s a pathway to impact that I give some weight to, but can’t discuss publicly due to concern that talking about the pathway would be an information hazard. If you’re interested in learning more about this consideration, shoot me an email.
(This consideration is only a small part of why I’m excited about psychedelics, and I’d still be bullish about psychedelics as an EA cause area in its absence.)
4. Neglectedness and tractability
How do psychedelics perform under the importance, neglectedness, tractability framework? The “importance” consideration is discussed above. Neglectedness and tractability are considered below.
4(a). Neglectedness
In short, psychedelic research is neglected. As discussed here, scientific research is very expensive, and current psychedelic research is happening only because a handful of private donors are funding it. (For more background, see this list of Open Phil analyses of the scientific research funding landscape.)
Governments are a major driver of scientific funding (NIH intends to grant $39.1 billion in 2019), and governments have not funded psychedelic research at all (presumably due to concern about optics, as well as bureaucracies being generally slow to update their stance).
From conversations I’ve had over the past two years, psychedelic research seems highly funding constrained, such that more money would convert into more research.
Roughly $40 million has been committed to psychedelic research since 2000.[4]
In comparison, the Open Philanthropy Project has granted $104 million to scientific research since 2011.
Another comparison point: Open Phil’s grant to establish CSET ($55 million) is larger than the total amount spent on psychedelic research in the last 20 years, worldwide.
4(b). Tractability
I’ve spent the last two years learning about the various projects going on in the psychedelic community. My general conclusion is that there are many tractable projects that could be happening but aren’t, due to lack of funding & capacity.
Here are the most important lines of psychedelic work currently happening:
- MDMA is being shepherded through the FDA approval process for PTSD treatment by MAPS.
- Psilocybin is being shepherded through the FDA approval process as a treatment for depression by both Compass Pathways and the Usona Institute (independently).
- State-level political campaigns to liberalize psychedelics are underway in Oregon & California. The city of Denver recently decriminalized psilocybin, by a narrow margin.
Both MDMA for PTSD & psilocybin for depression have received breakthrough therapy designation from the FDA, which indicates that the FDA thinks that these treatments “may demonstrate substantial improvement over available therapy.”
MDMA therapy is on track be made available as a prescription medicine in 2021 or 2022. Psilocybin therapy is on track to be made available as prescription medicine sometime between 2022 and 2025.
There’s a huge amount of work that needs to be done to make these rescheduling processes go well. There are insurance considerations, marketing considerations, regulatory considerations, and legal considerations that haven’t been figured out yet but need to be.
There isn’t an obvious place for psychedelic therapy in the current US healthcare infrastructure, so new infrastructure will need to be built (e.g. clinics for administering psychedelic therapy). There are opportunities to address this from both non-profit and for-profit approaches.
Furthermore, we still don’t granularly understand how psychedelics work neurologically, and we haven’t explored much of the space of possible applications of these substances. More research is needed – research groups at Johns Hopkins, NYU, Yale, the Heffter Research Institute, and Imperial College London all have deep research agendas they’d like to pursue, and they are all funding-limited.
Finally, as rescheduling approaches & psychedelics gain popularity, there’s a massive amount of public education work that needs to be done. The psychedelic experience is not without risk. As a greater number of inexperienced people have psychedelic trips, it’s important to propagate good guidelines for how to approach psychedelics in a safe & respectful way.
5. Conclusion
I’m offering three prizes for the best arguments against psychedelic research & advocacy being an EA cause area. Any argument submitted as a comment on this post will be considered for these prizes.
I’ll award $400 to the most-upvoted argument, $200 to the second-most-upvoted argument, and $100 to the third-most-upvoted argument. Winners will be assessed on Monday, June 3rd.
The driving motivation for this post is to surface arguments against psychedelic research & advocacy being an EA cause area. My current view is that psychedelics are an extremely promising altruistic cause area (on par with global health & x-risk reduction) – I’d like to learn more about how this might be mistaken.
If my view about psychedelics being a very promising cause area isn't mistaken, I’d like to see more attention paid to psychedelic research & advocacy by the EA community.
Cross-posted to LessWrong. Thanks to the Clarity Health Fund for a microgrant that enabled me to offer these prizes. This post was inspired by Grue_Slinky’s prize for the best argument against the EA Hotel.
[1]: More arguments in support of psychedelics being an EA cause area in these posts: Legal psychedelic retreats launching in Jamaica, Psychedelics Normalization, "A Psychedelic Renaissance" (Chronicle of Philanthropy)
[2]: Even for this case, you could imagine some scenarios where the intervention doesn't result in good outcomes.
To the extent there are not-good scenarios you could say it's not robust. It's probably best to model "robustness to cluelessness" as a continuum, wherein some interventions are more robust than others.
[3]: In correspondence with one of the study authors, it was revealed that some of the study participants were also given methamphetamine. The author said that back in the 1960s, they were considering methamphetamine to be similar to caffeine, so did not think much about administering it. Nevertheless, the study results are seriously confounded because of this.
[4]: Estimate confirmed via private correspondence.
[Own views]
0) I don't know what the bar should be for calling something a 'cause area' or 'EA interest' should be, but I think this bar should be above (e.g.) 'promising new drug treatment for bipolar disorder', even though this is unequivocally a good thing. Wherever exactly this bar falls (I don't think it needs to be 'as promising as global health'), I don't think psychedelics meet it.
1) My scepticism on the mental health benefits of psychedelics mainly rely on second-order causes for concern, namely:
1.1) There's some weak wisdom of nature prior that blasting one of your neurotransmitter pathways for a short period is unlikely to be helpful. This objection is pretty weak, given existing psychiatric drugs are similarly crude (although one of their advantages by the lights of this consideration is they generally didn't come to human attention by previous recreational use).
1.2) I get more sceptical as the number of (fairly independent) 'upsides' of a proposed intervention increases. The OP notes psychedelics could help with anxiety and depression and OCD and addiction and PTSD, which looks remarkably wide-ranging and gives suspicion of a 'cure looking for a disease'. (That they are often mooted as having still other benefits on people without mental health issues such as improving creativity and empathy deepens my suspicion). Likewise, a cause that is proposed to be promising on long-termism and its negation pings suspicious convergence worries.
1.3) (Owed to Scott Alexander's recent post). The psychedelic literature mainly comprises small studies generally conducted by 'true believers' in psychedelics and often (but not always) on self-selected and motivated participants. This seems well within the territory of scientific work vulnerable to replication crises.
1.4) Thus my impression is that although I wouldn't be shocked if psychedelics are somewhat beneficial, I'd expect them to regress at least as far down to efficicacies observed in existing psychopharmacology, probably worse, and plausibly to zero. Adding to the armamentarium of therapy for mental illness (in expectation) is worthwhile, but not enough for a big slice of EA opinion: it being a promising candidate for further exploration relies on 'neartermism' and (conditional on this) the belief that mental health is similarly promising to standard global health interventions on NTDs etc.
2) On the 'longtermism' side of the argument, I agree it would be good - and good enough to be an important 'cause' - if there were ways of further enhancing human capital. (I bracket here the proposed mental health benefits, as my scepticism above applies even more strongly to the case that psychedelics are promising based on their benefits to EA community members' mental health alone).
My impression is most of the story for 'how do some people perform so well?' will be a mix of traits/'unmodifiable' factors (e.g. intelligence, personality dispositions, propitious upbringing); very boring advice (e.g. 'Sleep enough', 'exercise regularly'); and happenstance/good fortune. I'd guess there will be some residual variance left on the table after these have taken the lion's share, and these scraps would be important to take. Yet I suspect a lot of this will be pretty idiographic/reducible to boring advice (e.g. anecdotally, novelists have their own peculiar habits for writing: IIRC Nabokov used index cards, Pullman has a writing shed, Gaiman a 'novel writing pen' - maybe 'having a ritual for dedicated work' matters, but which one is a matter of taste).
The evidence for psychedelic 'enhancement' is even thinner than psychedelic therapy, and labours under a more adverse prior. I agree the case for psychedelics here is comparable to CFAR/Paradigm/rationality training, but I would rule both out, not in.
3) I agree with agdfoster that psychedelics have reputational costs. This 'bad rap' looks unfair to me (notwithstanding the above, I'm confident that an 'MDMA habit' is much better for you than an alcohol, smoking, extreme sports, or social media one, none of which attract similar opprobrium), but it is decision-relevant all the same. If the upside was big enough, these costs would be worth paying, but I don't think they are.
The data doesn't support this, and generally suggests that 1-3 psychedelic experiences can have beneficial effects lasting 6 months or longer. See for example Carhart-Harris et al. 2018:
"Although limited conclusions can be drawn about treatment efficacy from open-label trials, tolerability was good, effect sizes large and symptom improvements appeared rapidly after just two psilocybin treatment sessions and remained significant 6 months post-treatment in a treatment-resistant cohort."
Griffiths et al. 2016:
"High-dose psilocybin produced large decreases in clinician- and self-rated measures of depressed mood and anxiety, along with increases in quality of life, life meaning, and optimism, and decreases in death anxiety. At 6-month follow-up, these changes were sustained, with about 80% of participants continuing to show clinically significant decreases in depressed mood and anxiety."
Johnson et al. 2017:
"All 15 participants completed a 12-month follow-up, and 12 (80%) returned for a long-term (≥16 months) follow-up, with a mean interval of 30 months (range = 16 – 57 months) between target-quit date (i.e., first psilocybin session) and long-term follow-up. At 12-month follow-up, 10 participants (67%) were confirmed as smoking abstinent. At long-term follow-up, nine participants (60%) were confirmed as smoking abstinent."
I would push back against the idea that these upsides are as independent as they may seem. Depression and anxiety are often comorbid (Hirschfeld 2001) and often comorbid with addiction (Quello 2005), OCD (Tukel 2002) and eating disorders (Marucci 2018). It seems that similar neurological states and cognitive processes underly these mental disorders, which is why psychedelics can effectively treat them all.
Carhart-Harris et al 2017, for example, suggest "connectedness" as the mechanism:
"A sense of disconnection is a feature of many major psychiatric disorders, particularly depression, and a sense of connection or connectedness is considered a key mediator of psychological well-being, as well as a factor underlying recovery of mental health. One of the most curious aspects of the growing literature on the therapeutic potential of psychedelics is the seeming general nature of their therapeutic applicability, i.e. they have shown promise not just for the treatment of depression but for addictions, anxiety and obsessive-compulsive disorder. This raises the question of whether psychedelic therapy targets a core factor underlying mental health. We believe that it does, and that connectedness is the key."
A secondary point here is that substances with different pharmacological and phenomenological effects are all grouped under the term "psychedelic". MDMA, for example, works and feels differently from ketamine, which works and feels differently from "classical" psychedelics like LSD, psilocybin, and DMT. So while it may seem unlikely that psychedelics (understood as one uniform thing) could have a range of benefits, it makes more sense when psychedelics are understood as a category that includes different substances.
I think that the wisdom of nature prior would say that we shouldn't expect blasting a neurotransmitter pathway to be evolutionarily adaptive on average. If we know why something wouldn't be adaptive, then it seems like it doesn't apply. This prior would argue against claims like "X increases human capital", but not claims like "X increases altruism", since there's a clear mechanism whereby being much more altruistic than normal is bad for inclusive genetic fitness.
I would worry about this more if the OP were referring to a specific intervention rather than a class of interventions. I think that the concern about being good on longterm and shortterm perspectives is reasonable, though there is a proposed mechanism (healing emotional blocks) that is related to both.
Normal drug discovery seems to be based off of coming up with hypotheses, then testing many chemicals to find statistically significant effects. In contrast, these trials are investigating chemicals that people are already taking for their effects. Running many trials then continuing the investigations that find significance is a good way to generate false positives, but that doesn't seem to be the case here, and I would be surprised to find zero effect (as opposed to shorter or different effects) if it were investigated more thoroughly.
I also think that improving human capital is important, and am not convinced that this is a clear and unambiguous winner for that goal. I'm curious about what evidence would make you more optimistic about the possibility of large improvements to human capital.
I think small studies are also more vulnerable to publication bias.
On the flip side, it may be possible that the "true believers" actually are on to something, but they have a hard time formalizing their procedure into something that can be replicated on a massive scale. So if larger studies fail to replicate the results from the small studies, this may be the reason why.
Do you have any examples of this actually happening? I have seen it as an excuse for things that never pan out many times, but I don't recall an instance of it actually delivering. E.g. in Many Labs 2 and other mass reproducibility efforts, you don't find a minority of experimenters with a 'knack' who get the effect but can't pass it on to others.
I don't have data either way, but "knacks" for psychotherapy feel more plausible to me than "knacks" for producing the effects in Many Labs 2 (just skimming over the list of effects here). Like, the strongest version of this claim is that no one is more skilled than anyone else at anything, which seems obviously false.
Suppose we conduct a study of the Feynman problem-solving algorithm: "1. Write down the problem. 2. Think real hard. 3. Write down the solution." A n=1 study of Richard Feynman finds the algorithm works great, but it fails to replicate on a larger sample. What is your conclusion: that the n=1 result was spurious, or that Feynman has useful things to teach us but the 3-step algorithm didn't capture them?
I haven't read enough studies on psychedelics to know how much room there is in the typical procedure for a skilled therapist to make a difference though.
Wouldn't 1.2), 1.3), and 1.4) point towards funding more psychedelic research?
(To prove or disprove the benefits found in the early-stage trials?)
It does, but although that's enough to make it worthwhile on the margin of existing medical research, that is not enough to make it a priority for the EA community.
Are you saying that EA shouldn't fund confirmatory research, in general?
Or are you saying that there's something in particular about this research, such that EA shouldn't fund confirmatory research in this case?
The latter. EA shouldn't fund most research, but whether it is confirmatory or not is irrelevant. Psychedelics shouldn't make the cut if we expect (as I argue above) we expect a lot of failure to replicate and regression, and the true effect to be unexceptional in the context of existing mental health treatment.
Got it, thanks!
I feel confused about why you think psychedelics shouldn't make the cut. The present state of research (several small-n studies finding very large effect sizes) seems consistent with both:
It seems like the only way to know which world we're in is to do confirmatory research.
That sounds a bit like the argument 'either this claim is right, or it's wrong, so there's a 50% chance it's true.'
One needs to attend to base rates. Our bad academic knowledge-generating process throws up many, many illusory interventions with purported massive effects for each amazing intervention we find, and the amazing interventions that we do find disproportionately were easier to show (with the naked eye, visible macro-correlations, consistent effects with well-powered studies, etc).
People are making similar arguments about cold fusion, psychic powers (of many different varieties), many environmental and nutritional contaminants, brain training, carbon dioxide levels, diets, polyphasic sleep, assorted purported nootropics, many psychological/parenting/educational interventions, etc.
Testing how your prior applies across a spectrum of other cases (past and present) is helpful for model checking. If psychedelics are a promising EA cause how many of those others qualify? If many do, then any one isn't so individually special, although one might want to have a systematic program of systematically doing rigorous testing of all the wacky claims of large impact that can be tested cheaply.
If not, then it would be good to explain what exactly makes psychedelics different from the rest.
I think the case for psychedelics the OP has made doesn't pass this standard yet, so doesn't meet the standard for an EA cause area.
From what I understand, effect size is one of the better ways to predict whether a study will replicate. For example, this paper found that 77% of replication effect sizes reported were within a 95% prediction interval based on the original effect size.
As a spot check, you say that brain training has massive purported effects. I looked at the research page of Lumosity, a company which sells brain training software. I expect their estimates of the effectiveness of brain training to be among the most optimistic, but their highlighted effect size is only d = 0.255.
A caveat is that if an effect size seems implausibly large, it might have arisen due to methodological error. (The one brain training study I found with a large effect size has been subject to methodological criticism.) Here is a blog post by Daniel Lakens where he discusses a study which found that judges hand out much harsher sentences before lunch:
However, I think psychedelic drugs arguably do pass this test. During the 60s, before they became illegal, a lot of people kind of were talking about how society would reorganize itself around them. And forget about performing surgery or driving while you are tripping.
The way I see it, if you want to argue that an effect isn't real, there are two ways to do it. You can argue that the supposed effect arose through random chance/p-hacking/etc., or you can argue that it arose through methodological error.
This is exactly what p-values are designed for, so you are probably better off looking at p-values rather than effect size if that's the scenario you're trying to avoid.
I suppose you could imagine that p-values are always going to be just around 0.05, and that for a real and large effect size people use a smaller sample because that's all that's necessary to get p < 0.05, but this feels less likely to me. I would expect that with a real, large effect you very quickly get p < 0.01, and researchers would in fact do that.
(I don't necessarily disagree with the rest of your comment, I'm more unsure on the other points.)
Yes, this is a better idea.
This comment is a wonderful crystallisation of the 'defensive statistics' of Andrew Gelman, James Heathers and other great epistemic policemen. Thanks!
I'm not claiming this. I'm claiming that given the research to date, more psychedelic research would be very impactful in expectation. (I'm at like 30-40% that the beneficial effects are real.)
I haven't read the literatures for all the examples you gave. For psychic powers & cold fusion, my impression is that confirmatory research was done and the initial results didn't replicate.
So one difference is that the main benefits of psychedelic therapy haven't yet failed to replicate.
> I'm at like 30-40% that the beneficial effects are real.)
Right, so you would want to show that 30-40% of interventions with similar literatures pan out. I think the figure is less.
Scott referred to [edit: one] failure to replicate in his post.
Scott referred to one failure to replicate, for a finding that a psychedelic experience increased trait openness. This isn't one of the benefits cited by the OP.
More on psychedelics & Openness:
Also:
I think we have a disagreement about what the appropriate reference class here is.
The reference class I'm using is something like "results which are supported by 2-3 small-n studies with large effect sizes."
I'd expect roughly 30-40% of such results to hold up after confirmatory research.
Somewhat related: 62% of results assessed by Camerer et al. 2018 replicated.
It's a bit complicated to think about replication re: psychedelics because the intervention is showing promise as a treatment for multiple indications (there are a couple studies showing large effect sizes for depression, a couple studies showing large effect sizes for anxiety, a couple studies showing large effect sizes for addictive disorders).
Could you say a little more about what reference class you're using here?