Wrong lessons from the FTX catastrophe

Excellent post. I hope everybody reads it and takes it onboard.

One failure mode for EA will be over-reacting to black swan events like this that might not carry as much information about our organizations and our culture as we think they do. 

Sometimes a bad actor who fools people is just a bad actor who fools people, and they're not necessarily diagnostic of a more systemic organizational problem. They might be, but they might not be. 

We should be open to all possibilities at this point, and if EA decides it needs to tweak, nudge, update, or overhaul its culture and ethos, we should do so intelligently, carefully, strategically, and wisely -- rather than in a reactive, guilty, depressed, panicked, or self-flagellating panic.

Downvoted. I disagree quite strongly on points one and four, but that's a discussion for another day; I downvoted because point three is harmful.

If people with a long history of criticising EA have indeed claimed X for a long time, while EA-at-large has said not-X; and X is compatible with the events of the past week, while not-X is not (or is less obviously compatible, or renders those events more unexpected); then rational Bayesians should update towards the people with the long history of criticising EA. Just apply Bayes' rule: if P(events of the last week | X) > P(events of the last week | not-X), then you should increase your credence in X upon observing the events of the last week.

This reasoning holds whether or not these critics are speaking in bad faith, have personal issues with EA, or are acting irrationally. If being a bad-faith critic of EA provides you with better predictive power than being a relatively-uncritical member of the movement, then you should update so that you are closer to being a bad-faith critic of EA than to being a relatively-uncritical member of the movement. You probably shouldn't go all the way there (better to stop in the middle, somewhere around 'good-faith critic' or 'EA adjacent' or 'EA but quite suspicious of the movement's leadership'), but updating in that direction is the rational Bayesian thing to do.

To be sure, there's always a worry that the critics have fudged or falsified their predictions, saying something vaguely critical in the past which has since been sharpened into 'Several months ago, I predicted that this exact thing would happen!' This is the 'predicting the next recession' effect, and we should be vigilant about it. But while this is definitely happening in a lot of cases, in some of the most high-profile ones I don't think it applies: I think there were relatively concrete predictions made that a crisis of power and leadership of pretty much this kind was enabled by EA structures, and these predictions seem to have been closer to the mark than anything EA-at-large thought might happen.

I think there is a further sense, that many EAs seem to feel that their error was less one of prediction than of creativity: it's less that they made the wrong call on a variety of questions, but simply that they didn't ask those questions. This is obviously not true of all EAs, but it is definitely true of some. In cases like this, listening more closely to critics - even bad faith ones! - can open your mind up to a variety of different positions and reasoning styles that previously were not even present in your mind. This is not always inherently good, of course, but if an EA has reason to think that they have made a failure of creativity then it seems like a very positive way to go.

For more context about my worries: I think that it is possible that OP might be including me, and some things I have tweeted, in point three. I have quite a small follower count and nothing I wrote 'blew up' or anything, so it's definitely very unlikely; but I did tweet out several things pretty heavily critical of the movement in recent days which very strongly pattern-match the description given above, including pointing out that prior criticisms predicted these events pretty well, and having relatively well-known EAs reaching out to me about what I had written. Certainly, I 'felt seen' (as it were) while reading this post.

I don't think I am a 'nefarious actor', or have a history of 'hating EA', but I worry that in some segments of EA (not the whole of EA - some people have gone full self-flagellation, but in some segments) these kinds of terms are getting slung around far too liberally as part of a more general circling-the-wagons trend. And I worry that posts like this one legitimise slinging these terms around in this manner, by encouraging the thought that EA critics who are engaging in some (sometimes fully-justified) 'told you so' are just bad actors trying to destroy their tribe. EA needs to be more, not less, open to listening to critics - even bad-faith critics - after a disaster like this one. This is good Bayesianism, but it's also just proper humility.

Strongly agree with all of these points.

On point 2: The EA movement urgently needs more earners-to-give, especially now. One lesson that I think is correct, however, is that we should be wary of making any one billionaire donor the face of the EA movement. The downside risk—a loss of credibility for the whole movement due to unknown information about the billionaire donor—is generally too high.

(Upvoted)

Events are not evidence to the truth of philosophical positions.

Are you sure? How about this position from Richard Chappell's post?

(3) Self-effacing utilitarian: Ex-utilitarian, gave up the view on the grounds that doing so would be for the best.

Psychological effects of espousing a moral theory are empirical in nature. Observations about the world could cause a consequentialist to switch to some other theory on consequentialist grounds, no?

Not sure there's a clean division between moral philosophy and moral psychology.

I agree hastily jumping to a different theory while experiencing distress seems bad, but it seems reasonable to update a bit on the margin.

I agree investigation should be thoughtful, but now seems as good as any opportunity to discuss. You say we should wait until facts are properly established, but I think discussion now can help establish facts, the same way a detective would want to visit the scene of a crime soon after it was committed.

+1 from me.

 I was talking about the whole situation with my parents, and they mentioned that their local synagogue experienced a very similar catastrophe, with the community's largest funder turning out to be a con-man. Everybody impacted had a lot of soul-searching to do, but ultimately in retrospect, there was really nothing they could or should have done differently—it was a black-swan event that hasn't repeated in the quarter of a century or so since it happened, and there were no obvious red flags until it was too late. Yes, we can always find details to agonize over, but ultimately, I doubt it will be very productive to change our whole modus operandi to prevent this particular black swan event from repeating (with a few notable exceptions).

I think many of these lessons have more merrit to them than you assume. To speak specifically about the ‘earning to give’ one, yes EA has pointed out that you should not do harm with your job to give it away. However I also think it is a bit psychologically naïeve to think that what happened with FTX is the last time that giving people the advice of earning to give is the last time it will lead to people doing harm to make money.

Trade-offs between ethical principles and monetary gain are not rare, and once we have established making as much money as possible (to give it away) as a goal in itself and something that gives status, it can be hard to make these trade-offs the way you are supposed to. It is not easy to accept a setback in wealth, power and (moral) status so lying to yourself or others to think that what you are doing is ethical becomes easy. It is also generally risky for individuals to become incredibly rich or powerful, especially if that depends on a misguided believe that some group membership (ea) makes you inherently ethical and therefore more trustworthy, since power tends to corrupt.

At the minimum I would like EA to talk more about how to jointly maximize the ethics of how you earn and spend your money, making sure that we promote people to gain their wealth in ways that add value to the world.

I disagree that events can't be evidence for or against philosophical positions. If empirical claims about human behaviour or the real-world operation of ethical principles are relevant to the plausibility of competing ethical theories, then I think events can provide evidential value for philosophical positions. Of course that raises a much broader set of issues and doesn't really detract from the main point of this post, but I thought I would push back on that specific aspect.

Brilliant post. Thanks for writing it. I just want to add to what you said about ethics. It seems that evaluating whether an action / event is good or bad itself presupposes an ethical theory.^[1] Hence I think a lot of the claims that are being made can be described as either (a) this event shows vividly how strongly utilitarianism can conflict with 'common-sense morality' (or our intuitions)^[2] or (b) trying to follow^[3] utilitarianism tends to lead to outcomes which are bad by the lights of utilitarianism (or perhaps some other theory). The first of these seems not particularly interesting to me, as suggested in your post, and the second is a separate point entirely - but is nonetheless often being presented as a criticism of utilitarianism.

1. ^{^}

   Someone else made this point before me in another post but I can’t find their comment.

2. ^{^}

   But note that this applies mostly to naive act utilitarianism.

3. ^{^}

   By which I mean 'act in accordance' with, but it's worth noting that this is pretty underdetermined. For instance, doing EV calculations is not the only way to act in accordance with utilitarianism.

I have to disagree with point 3. I think, due to imbalances in incentive structures, you sometimes have to take the "haters" criticisms into account. 

Take cryptocurrency and NFT's for example. Proponents of NFT's, who were financially invested, had a huge financial incentive to talk up their usefulness. They can make gargantuan sums of money from hype, and hence can full-time employ very smart people, fund their own media ecosystem, etc.  You can find any number of highly researched and polished literature talking up the 

But what if you (correctly imo) thought that NFT's  were a useless bubble? There were no big venture capitalists throwing out money for criticising NFT's, and shorting them was not really a valid financial path for a variety of reasons. Debunking NFT proponents is a lot of work, which doesn't make a lot of sense to do if you have a day job. 

But there is one incentive to critique NFT's that is powerful enough to motivate that kind of work and effort: Hatred of NFT's. This comes from someone who's been scammed, knows someone who's been scammed, or just pure "someone is wrong on the internet" energy. So of course they come off as "FUD" and haters with a dose of schadenfreude. The non-haters didn't bother writing lengthy critiques, they just shrugged, said it looked kinda dumb, and went on with their lives. 

I hope the analogies to FTX and EA are clear here. EA is highly obscure, with a similar financial incentive imbalance in place. If you want people to put actual effort into critiquing it, you either need to pay out more to skeptics (I fully support the critique competitions), or put up with some haters who may still have legitimate points.