(content warning: murder, medical abuse. this is a crosspost from LessWrong.)
A few years ago, my old middle school and another school in its district were renamed; they had originally been named after once-esteemed educational figures who were now in disrepute thanks in large part to their links to eugenics. I believe at the time I may have thought that it was odd that the schools had not been renamed sooner -- eugenics had already been discredited for decades at that point.
But at the time that these men lived, eugenics was not discredited. In fact, it was extremely popular. While eventually things changed, much of society was taken in first -- with terrible consequences. Unjust laws were passed based on eugenic principles in many nations, perhaps most notably in Nazi Germany. Hundreds of thousands of people were killed under government eugenics programs while hundreds of thousands more were involuntarily sterilized. Nowadays these acts are widely viewed as murder and abuse, but it took society some time to reach that conclusion and in the meantime things went pretty badly.
Worse still, eugenics is not the only such disgrace. You may well have heard of the lobotomy, an invasive medical procedure that is now widely viewed as destructive and abusive. However, when the lobotomy was first developed, its creator was not condemned as a medical abuser and removed from his position -- in fact, he received the Nobel Prize for his creation! Tens of thousands of people were lobotomized before lobotomies became viewed as abusive and were banned or greatly restricted in many jurisdictions.
Similarly, there was an extended period in the 1980s and 1990s where many scientists and media figures believed that techniques known as "recovered memory therapy" were helping people recover suppressed memories of abuse they had actually suffered as children -- now the general view is that this therapy was creating false memories, not bringing hidden true ones to the surface. Unfortunately, before recovered memory therapy was discredited its "revelations" led to many people being falsely accused of child abuse and in some cases even Satanic activity, and some had their reputations destroyed or were unjustly imprisoned on the basis of these "recovered" memories.
All three of these cases -- eugenics, lobotomies, and recovered memory therapy -- are instances where bad moral and medical thinking led to grave abuses. Worse still, all of them were popular within the last hundred years -- one doesn't exactly have to go back to the days of bloodletting and trepanation to find these sorts of errors.
To me, this seems to suggest a worrisome problem. How are we to know that we aren't making similar errors today? What can we do to try and protect ourselves from these sorts of mistakes?
There is a famous work called "Who Goes Nazi" that asks one to consider who among one's acquaintances would be likely to support the Nazi regime. (It was written in 1941 when this was very much a live concern.) I don't think rationalists or Effective Altruists would have been taken in much by the Nazis, but I worry that when it comes to whether we would have supported eugenics, many of us might have failed the test -- after all, this was very much scientific and medical orthodoxy at the time.
To put things another way, I think a lot of us wouldn't be fooled by demagogues but that scientific/medical misconduct might be something else. I'm curious whether anyone has especially good ideas or suggestions as to how best to think about these things -- ideally I would like to be able to say "EA is the kind of movement that wouldn't have supported eugenics even when many scientists and doctors did", but I don't feel confident in that statement at this time.
(I should clarify also that this is not meant as an insult, but rather as a case for moral examination and reflection.)
In general a key component of moral fashions seems to be the use of extreme social pressure to suppress dissent. Sadly this seems to be on the rise recently, especially on college campuses, and has even effected parts of the EA movement. Ironically, tolerating [what seem to many people] to be immoral beliefs, and debating them logically rather than stigmatizing and shunning their adherents seems to be the best strategy for avoiding genuine moral 'fads'. Paul Graham has a great article from 2004 about the importance of thinking about the things you're not socially allowed to say.
Somewhat unrelatedly, it seems not merely plausible but very likely that most EAs would have supported eugenics in the past. Perhaps they would not have supported the involuntary killing of adults as a component, but there are many types of eugenics which do not require this. Even today I think most EAs support the ability to abort babies for arbitrary reasons, including genetic problems.
Something that provides some amount of protection is the idea of ethical injunctions.
"Ethical injunctions are rules not to do something even when it's the right thing to do. (That is, you refrain "even when your brain has computed it's the right thing to do", but this will just seem like "the right thing to do".)"
The problem is, this doesn't help with some of these examples. Ethical injunctions would protect us against the first item. A 1940's EA might say "Yes, eugenics is a good thing, see the literature, but we still shouldn't kill or sterilise people without their consent to make it happen."
Lobotomies are a bit harder, but I think a suitably strong ethical injunction of "Don't perform surgeries without people's consent" would protect us there too. What it wouldn't protect us from would be thinking lobotomies were a good thing and managing to convince patients to let us do them.
Finally, ethical injunctions gives us no protection against recovered memory therapy. If you grant as a prior that RMT works, there is no harm being done - you are simply bringing old memories to light and bringing guilty people to justice.
So there's basically two separate problems here - how should EA avoid being on the wrong side of history morally, and how should EA avoid being on the wrong side of history scientifically?
I think our best bet for the first is to hold to universal moral principles like "Thou shalt not kill" and avoid allowing clever arguments to convince us to do things for the greater good. I think EA already does this pretty well - there's a strong norm against, say, lying about our top charities' effectiveness to solicit donations.
For the second one...I don't think there's a way out of that one. Science is humanity's best guess at the time, so we can't do better except in areas we might happen to have a comparative advantage in - psychology isn't one of them. All we can do is be open to changing our minds when the evidence stacks up against $WRONG_THEORY, so we should continue to promote epistemic norms of reasoning transparency and soliciting criticism of the movement.
I guess some scientific topics have some pretty good evidence and are hard to believe are extremely wrong (e.g. physics) given how much works so well that is based on it today, and then there are other scientific/medical areas that look scientific/medical without having the same robust evidence-base. I'd like to read a small overview meta analysis with some history of each field that claims (and is widely believed) to be scientific/medical, with discussion of some of its core ideas, and an evaluation of how sure we are that it is good and real in the way that a lot of physics is. I don't want to name particular other scientific/medical areas to contrast, but I do have at least one prominently in my mind.