Yeah, fully agree with this. I hope now that EV and/or EV-affiliated people are talking more about this matter that they'll be willing to share what specific due diligence was done before accepting SBF's gifts and what their due diligence policies look like more generally. 

There have been various calls for an independent investigation into the relationships between EA and SBF. Do you think such an investigation is warranted? Why or why not?

EV should have due diligence processes in place, instigated by EA's first encounter with a disgraced crypto billionaire/major EA donor (Ben Delo). 

In February 2021, CEA (the EV rebrand hadn't happened yet) wrote: 

Here’s an update from CEA's operations team, which has been working on updating our practices for handling donations. This also applies to other organizations that are legally within CEA (80,000 Hours, Giving What We Can, Forethought Foundation, and EA Funds).

• “We are working with our lawyers to devise and implement an overarching policy for due diligence on all of our donors and donations going forward.
• We've engaged a third party who now conducts KYC (know your client) due diligence research on all major donors (>$20K a year).
• We have established a working relationship with TRM who conduct compliance and back-tracing for all crypto donations.

Thanks for writing up these thoughts Will, it is great to see you weighing in on these topics.

I’m unclear on one point (related to Elizabeth’s comments) around what you heard from former Alameda employees when you were initially learning about the dispute. Did you hear any concerns specifically about Sam’s unethical behavior, and if so, did these concerns constitute a nontrivial share of the total concerns you heard? 

I ask because in this comment and on Spencer’s podcast (at ~00:13:32), you characterize the concerns you heard about almost identically. In both cases, you mention a bunch of specific concerns you had heard (company was losing money, Sam’s too risky, he’s a bad manager, he wanted to double down rather than accept a lower return), but they all relate to Sam’s business acumen/competence and there’s no mention of ethical issues. So I’m hoping you can clarify why there’s a discrepancy with Time’s reporting, which specifically mentions that ethical concerns were a significant point of emphasis and that these were communicated directly to you:

[Alameda co-founders wrote a document that] “accuses Bankman-Fried of dismissing calls for stronger accounting and inflating the expected value of adding new exchanges, and said a majority of employees thought he was “negligent” and “unethical.” It also alleges he was “misreporting numbers” and “failing to update investors on poor performance.” The team “didn’t trust Sam to be in investor meetings alone,” colleagues wrote. “Sam will lie, and distort the truth for his own gain,” the document says.

…Mac Aulay and others warned MacAskill, Beckstead and Karnofsky about her co-founder’s alleged duplicity and unscrupulous business ethics, according to four people with knowledge of those discussions. Mac Aulay specifically flagged her concerns about Bankman-Fried’s honesty and trustworthiness, his maneuvering to control 100% of the company despite promising otherwise, his pattern of unethical behavior, and his inappropriate relationships with subordinates, sources say.

Excellent points, everything you write here makes a lot of sense to me. I really hope you’re able to find funding for the proposal to research the EA brand relative to other alternatives. That seems like a really fundamental issue to understand, and your proposed study could provide a lot of valuable information for a very modest price. 

Thanks so much for this additional data and analysis! Really interesting stuff here. To me, the most interesting things are:

1. The midpoint of the satisfaction scale being a pretty good threshold for the point at which we see behavior changes 
2. The relatively high frequency of people with high satisfaction temporarily stopping promoting EA (and the general flatness of this curve)
3. I was surprised that for the cohort that changed their behavior, “scandal” was just one of many reasons for dissatisfaction and didn’t really stand out. The data you provide looks quite consistent with Luke Freeman’s observation: “My impression is that that there was a confluence of things that peaking around the FTX-collapse. There was building hostility towards some of the more avant garde EA ideas an actions of those associated with EA^[1] (towards both accurate and also misrepresentations of those ideas and actions) that seemed to get traction around just prior to the launch of WWOTF which meant there were a lot of people/opinions that got a lot more light when WWOTF was getting lots of attention and FTX failed so spectacularly. Then there was so much energy and angst in the system (both within the community and its critics) that I think the other issues compounded more than any individual one would have. The confluence of all this has sadly left a bad taste in a lot of people's mouths that costs fairly uncontroversially good things a lot in terms of action and advocacy.”

That makes sense. That said, while it might not be possible to quantify the extent of selectin bias at play, I do think the combination of a) favoring simpler explanations and b) the pattern I observed in the data makes a pretty compelling case that dissatisfied people being less likely to take the survey is probably much more of an issue than dissatisfied people being more likely to take the survey to voice their dissatisfaction.

As I mentioned in point 3 of this comment:

Looking at the distribution of satisfaction scores in late 2022 vs late 2023, we see the reduction in satisfaction over that period coming from fewer people giving high ratings of 9 or 10, and more people giving low ratings of 3-5. But for the lowest ratings, we see basically no change in the number of people giving a rating of 2, and fewer people (almost nobody) now giving a rating of 1.

This suggests we could crudely estimate the selection effects of people dropping out of the community and therefore not answering the survey by assuming that there was a similar increase in scores of 1 and 2 as there was for scores of 3-5. My guess is that this would still understate the selection bias (because I’d guess we’re also missing people who would have given ratings in the 3-5 range), but it would at least be a start. I think it would be fair to assume that people who would have given satisfaction ratings of 1 or 2 but didn’t bother to complete the survey are probably also undercounted in the various measures of behavioral change. 

Great post! Another excellent example of the invaluable work Rethink Priorities does.

My observations and takeaways from this latest survey are:

1. Overall satisfaction is low. On a 10 point scale, a 6.77 rating isn’t very good. Only ~12% give a rating of 9 or 10, and <40% give a rating of 8 or higher. (I’m aware that a 6.8 rating on a ten point scale still suggests more satisfaction than dissatisfaction, but given that we're talking about people rating their satisfaction with a community they've chosen to participate in I think we should interpret this result as damning with faint praise).
2. The change in satisfaction from pre-FTX to now is quite large. The original survey in the immediate aftermath of FTX’s collapse found a substantial drop in satisfaction vs. pre-FTX. One commenter at the time observed that “[decreased satisfaction by 0.5-1 points on a 10-point scale is] a big drop! In practice I've only ever seen this type of satisfaction scale give results between about 7/10 through 9.5/10… so that decline is a real big chunk of the scale's de facto range.” This latest survey suggests that the satisfaction has dropped another ~.2-.4 points. That means from pre-FTX to today, satisfaction has probably dropped something like .7-1.4 points, which is huge for a 10 point scale with a much narrower effective range (and the drop could easily be understated due to selection bias). The mean reversion of satisfaction to pre-FTX levels that some expected has not happened. 
3. Selection bias is probably causing us to overstate satisfaction, and understate the degree to which satisfaction has dropped.  As the report notes, selection bias could plausibly cut both ways. That said, I think most people would guess that selection bias is operating in the direction of excluding the least satisfied members from the data set. And the data seems to give some indication to support that view. Looking at the distribution of satisfaction scores in late 2022 vs late 2023, we see the reduction in satisfaction over that period coming from fewer people giving high ratings of 9 or 10, and more people giving low ratings of 3-5. But for the lowest ratings, we see basically no change in the number of people giving a rating of 2, and fewer people (almost nobody) now giving a rating of 1. Given falling satisfaction, we should expect more people to be giving these lowest ratings; the fact that we don’t suggests that the least satisfied people are excluded from the sample. 
4. The behavioral changes described in this survey are significant and warrant attention. The number of people making concrete behavioral changes surprised me (even though I already believed FTX had a significant negative impact on community growth and activity). I was particularly surprised at how many people reported permanently ceasing to promote EA or engage in key EA activities, especially since selection bias likely suppresses the reported numbers. The December 2022 survey indicated a community desire for additional transparency, reflections, and post-mortems. This more recent survey suggests the community has been unimpressed by the limited public efforts on these fronts, with concrete negative implications.
5. This analysis shows large drops in satisfaction and significant behavior changes, but only gives limited insight into what caused these. Issues like JDEI concerns and concerns about cause prioritization are clearly reasons why people might be dissatisfied, but it’s not obvious why these longstanding issues should cause people to be more dissatisfied now than they were in late 2022. We can hypothesize that the changes were driven by FTX (and to a lesser extent subsequent scandals), but it would be preferable to look directly at the data. I think it would be very valuable to closely examine the cohort of people who report having changed their behavior. (It would also be interesting to look at a cohort of people who report significant decreases in satisfaction, but it seems more important to understand drivers of behavior change.) For this cohort, I would like to know:
   1. What are the largest causes of dissatisfaction? How does this cohort compare to the full sample?
   2. Does dissatisfaction appear to be driven by one major factor, or a confluence of concerns (as Luke Freeman has suggested might be the case)?
   3. How much did reported satisfaction decrease since previous surveys?
   4. What levels of satisfaction [or sizes of decreases in satisfaction] are associated with behavior change (i.e. can we say something like “if satisfaction drops to 6 or lower [or decreases by 1 or more points], that’s when we start seeing people shift behaviors”?)
   5. How does perception of issues for this cohort compare to the full sample?
   6. Do perceptions of issues look different for modest behavior changes (e.g. stopping referring to “EA” but still promoting) vs. larger behavior changes (e.g. permanently stopping promoting)?
   7. Do we have any insights into the temporarily stopped promoting cohort (a sizeable 15% of the sample)? Do we have any idea whether these are people who stopped promoting for a few months but are now promoting again vs. people who are still not promoting but expect to do so again at some point? I realize the most recent survey might not give enough information to tell, but this seems like a very important issue to understand and something we might want to dig into in the full EA Survey.
   8. How much overlap is there across the different types of behavior change? 
6. Updating analyses of community growth seems like it should be a high priority. The last public effort was based on data collected in August 2023. I’ve been a longstanding proponent of conducting regular analyses of community growth, but refreshing that analysis (perhaps with some methodological tweaks) seems particularly important given the results of this survey. Various metrics that are publicly available don’t look great, continuing trends I first noted in October (though note that some of the metrics that performed strongest in the immediate aftermath of FTX, most notably 80k’s metrics, aren’t publicly available):
   1. Donations and donors to EA Funds continue to shrink year over year. The decline in donations has been a multi-year trend, but the decline in donors represents a reversal of a multi-year growth trend , and the timing of that reversal aligns perfectly with FTX’s collapse.
   2. GWWC donations and donors were down in 2023 vs. 2022, and pledges in Dec 23/Jan 24 were also down vs Dec 22/Jan 23 even though that latter period immediately followed FTX’s collapse and GWWC was “instructed to pause all our giving season campaigns around the time of the crisis.”
   3. EA Forum metrics, which had experienced years of consistent and rapid growth leading up to FTX’s collapse, have either flatlined at the levels they were at 2 years ago or have been steadily falling since the collapse (even as resources invested in the forum have increased significantly since then).
   4. Per CEA’s dashboard, the last few months have seen the lowest traffic ever to the effectivealtruism.org intro page (in a data series going back to 2017) and the EA newsletter subscription base has shown no sign of regaining the growth trajectory it was on for 2 years prior to FTX’s collapse; since the collapse subscribers have actually shrank slightly.
   5. As Jason astutely observed, the massive increase in public awareness around AI over the last year should have provided a significant tailwind to many metrics. I would argue the same is true for the hugely successful promotion around WWOTF. And demographic factors should generally support an expectation of growth in donation related metrics (almost everyone in the EA community is at an age where they are entering the workforce or increasing their earnings; virtually nobody is at retirement age). The weak performance we’ve observed in publicly available metrics looks even worse in this context.

Me too