Hi, this is something we’re already exploring, but we are not in a position to say anything just yet.
It doesn't quite ring true to me that we need an investigation into what top EA figures knew. What we need is an investigation more broadly into how this was allowed to happen. We need to ask:
It's not totally unreasonable to ask what EA figures knew, but it's not likely that they knew about the fraud, based on priors (it's risky to tell people beyond your inner circle about fraudulent plans), and insider reports. (And for me personally, based on knowledge of their character, although obviously that's not going to convince a sceptic.)
I strongly agree with this. In particular, it seems that the critiques of EA in relation to these events are much less focused on the recent fraud concern than EAs are in their defenses. I think we are choosing the easiest thing to condemn and distance ourselves from, in a very concerning way. Deliberately or not, our focus on the outrage against recent fraud distracts onlookers and community members from the more serious underlying concerns that weigh more heavily on our behavior given their likelihood.
The 2 most pressing to me are the possibilities (i) that EAs knew about serious concerns with FTX based on major events in ~2017-2018, as recently described by Kerry Vaughan and others, as well as more recent concerns, and (ii) that EAs acted as if we had tens of billions committed for our projects even though many of us knew that money was held by FTX and FTX-affiliated entities, in particular FTX Token (FTT), a very fragile, illiquid asset that could arguably never be sold at anywhere close to current market value and arguably makes statements of tens of billions based on market value unjustified and misleading .
[Edit: Just to be clear, I'm not referring to leverage or fraud with point (ii); I know this is controversial! Milan now raises these same two concerns in a more amenable way here: https://forum.effectivealtruism.org/posts/WdeiPrwgqW2wHAxgT/a-personal-statement-on-ftx?commentId=3ZNGqJEpQSrDuRpSu]
For what it's worth, as someone saying in another thread that I do think there were concerns about Sam's honesty circulating, I don't know of anyone I have ever talked to who expressed concern about the money being held primarily in FTT, or who would have predicted anything close to the hole in the balance sheet that we now see.
I heard people say that we should assume that Sam's net-wealth has high-variance, given that crypto is a crazy industry, but I think you are overstating the degree to which people were aware of the incredible leverage in FTX's net-position (if I understand the situation correctly there was also no way of knowing that before Alameda's balance sheet leaked a week ago. If you had asked me what Alameda's portfolio consists of, I would have confidently given you a much more diversified answer than "50% FTT with more than 50% liabilities").
That makes sense.
Hmm, I do think in the absence of the leverage, having wealth in FTT was kind of reasonable, and the leverage was the primary thing that enabled the whole thing to implode this quickly.
I was still surprised by Alameda not having a more diversified portfolio, but I think it's basically accurate to model FTT as stock, and it's not that crazy to have a lot of your wealth in your own stock (and for it to be hard for you to exit that position, since it looks really suspicious if you sell a lot of your own stock).
But I do agree that there was probably still too much reliance on assuming the FTX money would stay real. But like, I do think if you model FTT as stock, as I think it was straightforward to model as at the time, then I don't think there is really anything going wrong by saying that Sam had that much money, since like, it's pretty standard to calculate net-worth that way.
Yeah, there seems to be some confusion. Obviously a unicorn founder is going to have a lot of wealth in that company's stock (or close analogues such as FTT). The problem, rather, is that we think FTT was being transferred between corporate entities and used as collateral at its face value (there may've been other problems relating to FTT, such as its tokenomics, as well).
Matt Levine suggests that the key problem is accepting your own stock as collateral:
Now let’s add one more crypto element. If you are a crypto exchange, you might issue your own crypto token. FTX issues a token called FTT. The attributes of this token are, like, it entitles you to some discounts and stuff, but the main attribute is that FTX periodically uses a portion of its profits to buy back FTT tokens. This makes FTT kind of like stock in FTX: The higher FTX’s profits are, the higher the price of FTT will be. 8 It is not actually stock in FTX — in fact FTX is a company and has stock and venture capitalists bought it, etc. — but it is a lot like stock in FTX. FTT is a bet on FTX’s future profits.
But it is also a crypto token, which means that a customer can come to you and post $100 worth of FTT as collateral and borrow $50 worth of Bitcoin, or dollars, or whatever, against that collateral, just as they would with any other token. Or something; you might set the margin requirements higher or lower, letting customers borrow 25% or 50% or 95% of the value of their FTT token collateral.
If you think of the token as “more or less stock,” and you think of a crypto exchange as a securities broker-dealer, this is completely insane. If you go to an investment bank and say “lend me $1 billion, and I will post $2 billion of your stock as collateral,” you are messing with very dark magic and they will say no. 9 The problem with this is that it is wrong-way risk. (It is also, at least sometimes, illegal.) If people start to worry about the investment bank’s financial health, its stock will go down, which means that its collateral will be less valuable, which means that its financial health will get worse, which means that its stock will go down, etc. It is a death spiral. In general it should not be possible to bankrupt an investment bank by shorting its stock. If one of the bank’s main assets is its own stock — is a leveraged bet on its own stock — then it is easy to bankrupt it by shorting its stock.
Well, my understanding now is that it is very structurally different (not just reputationally or culturally different) from publicly traded stock: the tiny trading volume, the guaranteed price floor, probably other things. If it were similar, I think I would probably have much less of that concern. This does imply standard net worth calculations for Sam Bankman-Fried were poor estimates, and I put a decent chance on Forbes/Bloomberg/etc. making public changes to their methodology because of this (maybe 7% chance? very low base rate).
I've updated a little toward this being less concerning. Thanks.
I think the above points hold. No question that FTX transfering funds to Alameda is the crux of the moral issue at play here. But financially speaking, and to flush out the liquidity issue... for such significant holdings of FTT, those funds should have only been considered with a massive haircut, maybe even upwards of 90%. Alameda said they had billions worth of FTT (in USD terms), and sure that was the case at market prices prior to all of this this. But as we know from what precipatated everything, sell pressure in that range (as was about to happen from Binance), absolutely nuked the market's confidence and therefore value in FTT. So how much more so would this have been the case if FTX/Alameda ever had came along and said they needed to liquidate billions in FTT?
There was never going to be a way out. It's truly sad to see the way moral lines were crossed with the transfer of funds across entities, yet to me it's just bizarre that a bunch of seemingly brilliant folks would have organized their books in such a financially precarious manner when it came to liquidity... let alone in their own token.
Would you apply the same reasoning to eg Dustin's Asana stock? Or Bezos' Amazon stock? If not, why?
The same theme holds but still fairly different scenarios. Asana/Amazon stock signifies partial ownership of those companies derived from the value of their future cash flows. FTT was just an invented token trading on the confidence of FTX, but with no intrinsic value, hence a greater probability of it nuking to 0/
Overall though, Dustin/Bezos' holdings in each respective company are likely to warrant significant haircuts on their personal balance sheets. Rule of thumb is the more concentrated ownership, the higher the haircut, and the less liquidity, the higher the haircut. They both have significant ownership, so even though it's fairly safe to estimate higher liquidity in public markets, their concentration of ownership warrants a significant haircut.
I think what's so significant about the FTX/Alameda case tho, and why the FTT should have been considered effectively worthless on their balance sheet, even given the high market price, was the massive concentration of self-ownership (not to mention the Binance concentration risk) AND the very low liquidity.
FTT was just an invented token trading on the confidence of FTX, but with no intrinsic value, hence a greater probability of it nuking to 0/
Ah! My understanding was that FTX bought back some amount of FTT with their profits each week, giving it intrinsic value (conditional on FTX continuing to exist) dependent on FTX's future cash flows, and making it feel fairly analogous to stock to me. Though I'm not aware of the exact mechanism, and can easily imagine eg the amount of money going into the pool being independent of the total amount of FTT issued, and the total amount issued being non transparent, making the actual intrinsic value ~0 (essentially really diluted stock)
Good call and looks like they were fairly trainsparent with most of the info you're referencing here.
I guess I'm still not sure at what point, even in a best case scenario, the "buy and burn" mechanism actually would drive intrinsic value in the coin though? From a pure supply and demand perspective maybe, since the better FTX does, the more scarce the coin is and the more demand there is for it... but isn't that just a sort of strategic market manipulation within the exchange?
Albeit that could have worked had FTX reached a point where it was strong enough, but I think they effectively nullified that opportunity by using so much FTT as collateral that needed to be more liquid while treating it as if it was.
This seems directionally right to me. My current view is that an investigation should stick to objective questions rather than normative stuff about, for example, whether the EA movement is in principle OK taking funding from the crypto industry (or some other potentially destructive industry). I think my thinking here is that a person who isn't involved in EA should think about those objective questions, and then folks in the EA movement itself should make arguments for and decide on the future direction in response. I trust a lawyer specialised in independent investigations to do the fact-finding work and the 'what went wrong' work, but not to make principled judgements about what EA should think or do more broadly.
It’s definitely true that there are more philosophical questions that a lawyerly investigation wouldn’t be well-positioned to answer. But it seems likely that there were plenty of legal and financial risk-management mistakes that EA orgs made in the pst year that an independent investigator or other outside risk management consultant would be well-positioned to opine on.
This sounds like a sort of EA congressional investigation. An important part for that, which I don't know how to make happen, is to make sure the investigators are accountable to EA while not being controlled by the "executive branch" which is being investigated. One would think some kind of democratic processes would help.
A meta level structural problem may be that so much decision making seems to be focused on a relatively small group of people without much oversight. Even with the best people in the world that's going to lead to group think and blind spots. Other charities and non-profits have extensive oversight systems that may be worth imitating.
Much more sensible.
Let me kick off on point 4: I don't think it is accidental that EA's philanthropic model of social change leads to the concentration of power in the hands of a few people with outsized riches, or control over those riches. Why has the movement never even thought of democratising itself? Recall that JS Mill, being the utilitarian he was, argued for democracy on instrumental grounds - mostly because it led to better epistemic and cultural outcomes.
I disagree:
Wherever there is money, there will be fraud. That's why we have financial regulations, and why professional charities have accountants and auditors.
This could be a fatal blow for any charity that had become dependent on SBF's funds... which is exactly why GiveWell already has a rule that limits any single donor from giving more than (I think) 50% of funding for any one cause.
Rationally, the only way in which the EA community should change is in taking a deeper look at whether the key EA figures that have taken SBF's money could have been his "partners" infraud.
Even if this possibility is small, a professional investigation should happen because there is a non-trivial risk that this had happened
FWIW I don't know why you're being disagreement voted, I broadly agree. I think the money amounts at play here are enough to warrant an investigation even with a low possibility of uncovering something significant.
Thank you.
I'm certain that the disagreement votes comes:
As an outsider to the movement, I think this is misjudged.
I think it incredibly unlikely that SBF disclosed his fraudulent behavior to anyone outside of a small inner circle within FTX/Alameda. Why would he take that stupendous risk?
The failure here is becoming so dependent upon, and promoting the virtues of, someone engaged in a crypto business with a lot of red flags. In my opinion, that is what merits 'review'.
Investigating your own personnel for something you have no probable cause for will only consolidate the bad publicity EA is getting now. It will make EA look guilty for something it did not do.
If I am being honest, this comes across as an over-the-top attempt at self-cleansing that is motivated more by prim sanctimony than any real reckoning with the situation.
Thanks Geuss.
The failure here is becoming so dependent upon, and promoting the virtues of, someone engaged in a crypto business with a lot of red flags. In my opinion, that is what merits 'review'.
I agree, without prejudging what the answer to that would be. I think it's very likely that EA will do some soul searching, although this is a moral question rather than a question of fact, and so my sense is that it's probably less suitable for an independent fact-finding review.
Investigating your own personnel for something you have no probable cause for will only consolidate the bad publicity EA is getting now. It will make EA look guilty for something it did not do.
If I am being honest, this comes across as an over-the-top attempt at self-cleansing that is motivated more by prim sanctimony than any real reckoning with the situation.
I do care about the optics of EA, but I am more concerned for the epistemic and moral health of the community. For me, this is about taking seriously the fact that EA was in the orbit of a very bad thing that has caused significant suffering. I realise that this comes across differently if you see EA as being a less earnest movement than I do. There possibly is some reputational cost (which may or may not be outweighed on net by reputational gains) to doing this, but the reason for doing it largely sits outside some kind of optics calculus.
But in what domain is it good practice to set up a formal investigation of persons about whom there is no knowledge of wrong-doing, and in which the alleged crime - knoweldge of a second-hand crime - would be almost impossible to independently verify anyway. And, in doing so, permanently tarnishing the movement, and its ability to effectively operate in the future. That is a big gamble for... what reasonable gain?
One counterfacutal I think is worth considering: had Sam never loaned customer deposits to Alameda, how do you think everyone should have acted?
Had the loans never happened, FTX would still have been engaged in some fairly disreputable business, Sam would still have a wildly high appetite for risk, and just about all of the "red flags" people bring up would still have been there. However, even if this was all common knowledge, my best guess is that most people would've readily endorsed continuing to work with FTX and would not have endorsed making bureaucratic requirements too onerous for FTX funded projects. I think, even in this counterfactual, it might still have made sense to insist on FTX improving their governance before they further scale up their engagement with EA (and perhaps a few other things too).
I suspect that factually, whatever people reasonably could have known was most likely limited to "disreputable business and red flags", not that the loans to Alameda had happened. Furthermore, I doubt anyone even had particularly good reason to think FTX might be engaged in outright fraud on this scale - I think crypto exchanges go bust for non-fraudulent reasons much more often than for fraudulent ones. For these reasons, I suspect that while there are improvements to be made, they probably won't amount to drastic changes. I also suspect that, despite numerous negative signs about FTX, even insiders would have been justified in placing relatively little credence in things playing out the way they have.
As a brief addendum, I imagine in the non fraudulent world, Sam’s net worth is substantially smaller. So maybe the extremely fast growth of his wealth should itself be regarded with suspicion?
Yes, more broadly, I think that we should think about governance more… I guess there are a bunch of low-hanging fruits we can import from the broader world, e.g., someone doing internal-to-EA investigative journalism could have unraveled risks related to FTX/Alameda leadership or just did an independent risk analysis (e.g., this forecasting question put the risk of FTX default at roughly 8%/yr — I am not sure betters had any private information, I think just base-rates give probability around 10%).
Jehan gives some additional suggestions I liked here. Including rules about:
Might not have affected things in the FTX case, but perhaps worth considering whilst the window for significant reform is wide open.
Another structural question that will need answering at some point: Did anybody outside of FTX consider it okay that all of the directors at the FTX Foundation were senior FTX employees? Why were there no independent (of FTX) directors there?
Thanks for posting this!
My impression is that when a large company has a big scandal like this, a good practice is to do some investigation like you describe.
It seems safe to spend time and money just getting a really good handle on what happened, and who else might have been responsible.
I feel kind of sad that you, the author, feels worried posting this. I think that "suggesting that EA do good practiced of the business world" should be a very low bar for what we should be comfortable with.
----
(Aside, On Conflicts of Interest)
I'd note that in our situation, this will be messier that if we were all one large organization. All the non-OP orgs you mentioned above are financially dependent on Open Philanthropy (especially now with FTX out of the picture), and many of their board members are dependent on OP.
I'm a board member of Rethink Priorities, and my own org is likely dependent on Open Philanthropy in the future.
All this to say, the most obvious people to lead this, are the leaders and board members at Open Philanthropy. Others could come in, but do keep in mind that many members would likely not be a position to be objective of issues around Open Philanthropy at least.
It's like, if we were in a big company, and the CFO were have found to commit fraud - the people responsible to lead an investigation would be company board members - not really employees. Employees would have conflicts of interest. They could help out and provide evidence, but I assume their help would be limited.
I would be interested in the other main funders (maybe LongView and Jaan Tallinn) helping out. They might be the least dependent groups on OP around.
That said, I could imagine help from orgs like RP and CEA being pretty useful, but I'd probably lean more to groups without as much of an affiliation to OP.
I'm updating towards the view that there are two different tasks - first, a fact-finding exercise and second, decisions about how to respond.
It doesn't seem appropriate to me that Open Philanthropy directly conducts the fact-finding exercise - it seems important to have an independent person(s) who are able to receive confidential feedback. It does seem appropriate to me, though, for Open Phil to commission that investigation/inquiry though.
I don't know how EA should decide how to decide how to respond. I'm not sure that the financial dependence on OP is as much of an issue in this context, but agree it's probably worth addressing.
This is even more important than people think, since this may not be the first time EA had questionable relationships with millionaires/billionaires. I consider Kerry Vaughan's tweets about it plausible because this is a common theme of failure: small failures are normalized or rationalized, this allowing bigger failures to happen, like this fraud.
Here's links to normalization of deviance and Kerry Vaughan's tweets:
https://twitter.com/KerryLVaughan/status/1590807597011333120
Which millionaires/billionaires are you thinking of? I'm thinking of:
- Ben Delo
- Avraham Eisenberg
I don't think Avraham Eisenberg was ever touted in EA like Delo or SBF were and I'm not even sure if he self-identifies as an EA or makes EA-related donations.
Not directly related to the main point here, but Vaughan has commented on some of the points in this thread on the forum and the replies add some additional information and context to the 2018 SBF stuff from other people involved at the time:
Yes, it wasn't outright fraud, but this is still a bad sign for worse things to come.
Oh definitely, that’s why I wanted to emphasize that my point wasn’t directly relevant to the one you were making, I just thought it would be useful context for anyone who read the Twitter thread but not the ensuing discussion that clarified or nuanced some of the points it raises.
Posting quickly, haven't thought it through but guessing that one of the key benefits of having an independent investigator is that it gives routes for any evidence to be evaluated, thereby preventing a witch-hunt which could happen if no legitimate routes for recourse.
There is nothing wrong with a"did anybody know" approach and you, under any circumstances, should not be pressured away from pushing that line of inquiry and questioning, even by influential EAs. The rational move is to be skeptical of (though not accusatory towards) anyone who doesn't wish to investigate that possibility and what it might implicate.
If they have nothing to hide and did not break anyone's trust, they have nothing to fear.
To clarify, my update isn't that I think there shouldn't be a "did anybody know" aspect to the investigation - but rather that a broader fact-finding review would be useful, and that my original suggestion was probably too narrow.
In the coming months, it could be valuable to estimate the total EV of this fraud so we can 1) map actions to consequences and 2) more deeply understand what happened here.
That might include:
-Loss of life from direct victims
-Reputation damage to EA and subsequent loss of funding, talent, and reduced growth
-Reduced efficacy of existing EA orgs due to damaged trust, reduced resources, and trauma
Some of these will be difficult to estimate. But I think we should know what we lost, and those engaged in fraud should know what it cost.
Yeah not sure EV is necessarily the right perspective, but I see where you're going and think it's an interesting thought experiment. Considering how SBF's decision making and largely linear utility lead to high risk appetetite, this might have led him to underestimate harmful tail risk considering the global stage he was maintaining for EA.
It might be a big leap of faith, but assuming he justified some his actions with an ends justify the means attitude (with each marginal dollar gained having roughly equal utility to help others) , at what point tho did his social stature warrant the downside consideration of reputational damage for him and EA? When it was him early on at Alameda, a coin-flip with enough upside might have made sense since he wasn't socially influential enough to be risking reputational damage, just his net worth.
But heading into FTX, led alone into lawmaker's offices and every news outlet imaginable, he became a steward of EA in an unprecedented manner.. but it does not appear that maintaining his stature in a risk-reduced manner given his importance ever factored into his decision making.
This comment is being downvoted, but I think it's probably a worthwhile exercise (even though I expect it to be really net negative)
I see this as a last resort procedure, if there is also an evidence linking people. But doing this without a credible evidence is in effect the witch hunting that most are avoiding as it may destroy the EA community instead of get the pieces together.
I like the sentiment but disagree. We have to know how far this goes. This is a system failure, not just an individual failure.
Thanks Miguel. I am worried about a witch hunt too, although I think it's unlikely that a carefully conducted investigation that adheres to the principles of natural justice would destroy the EA community. My sense is that if the EA community is so weak that such an investigation would destroy the community, then the EA community will be toast anyway from the hit to community trust that is resulting/will result from the FTX situation.
Your suggestion is that there only be an investigation if some piece of evidence emerges. On first glance, my lawyer brain wants to agree with this. But if two weeks ago you had posed the hypothetical question to me "FTX has collapsed and SBF/others have misappropriated user funds; what is the probability that someone else in EA knew about it?", I think my answer would probably have been higher than 1% or 2%. For a movement that wants to positively affect the trajectory of humanity (and to be taken seriously by governments and civil society), I think that even 1% or 2% is high enough to justify an (carefully conducted, natural justice-oriented) investigation.
I see your point now Tyronne. It is to ensure that EA on itself will take its responsibility on assuring the public that it is not in any way a fraud committed with EA involvement. I am agreeing on you on this one.
There is some possible optics benefit to it, yes, in that we could point to this action in (careful, measured) response to future criticism of EA from this angle. But I think the much more important reason to do it is for own health as a community of do-gooders. Even if we didn't expect criticism, I would still want to prevent someone who was so careless from making decisions about the trajectory of the movement or about grant allocation etc (at least initially; expressing no view about whether such a person could redeem themselves, as that seems pretty fact specific and I'm not in a good position to offer a view anyway).
I'd argue the major point of an investigation is to establish the evidence
Any healthy community or organisation would, of course, do this without hesitation. My prediction, however, is that this thread will get downvoted and/or ignored.
I think this comment helped this post gain attention and made me more likely to engage. Thank you, Markus, for encouraging us to pay attention.
To be frank, that was the purpose of my comment. I still think it's a reasonable prediction based on previous posts, such as those pointed out by Tyrone in the original post, not leading to meaningful mechanisms such as a formal whistleblower function within EA. But I did make the reply pointed to get a reaction, and I hope this is not construed as dishonest in any way. Judging by all the disagreement votes, the upvotes for the original post, and the discussion going on here, I hope to be very, very wrong this time.
I think that key EA orgs (perhaps collectively) like the Center for Effective Altruism/Effective Ventures, Open Philanthropy, and Rethink Priorities should consider engaging an independent investigator (with no connection to EA) to try to identify whether key figures in those organisations knew (or can reasonably be inferred to have known, based on other things they knew) about the (likely) fraud at FTX.
The investigator should also be contactable (probably confidentially?) by members of the community and others who might have relevant information.
Typically a lawyer might be engaged to carry out the investigation, particularly because of professional obligations in relation to confidentiality (subject to the terms of reference of the investigation) and natural justice. But other professionals also conduct independent investigations, and there is no in principle reason why a lawyer needs to lead this work.
My sense is that this should happen very promptly. If anyone did know about the (likely) fraud at FTX, then delay potentially increases the risk that any such person hides evidence or spreads an alternative account that vindicates them.
I'm torn about whether to post this, as it may well be something that leadership (or lawyers) in the key EA orgs are already thinking about, and posting this prematurely might result in those orgs being pressured to launch an investigation hastily with bad terms of reference. On the other hand, I've had the concern that there is no whistleblower protection in EA for some time (raised in my March 2022 post on legal needs within EA), and others (e.g. Carla Zoe C) have made this point earlier still. I am not posting this because I have a strong belief that anyone in a key EA org did know - I have no information in this regard beyond vague speculation I have seen on Twitter.
If you have a better suggestion, I would appreciate you sharing it (even if anonymously).
Epistemic status: pretty uncertain, slightly anxious this will make the situation worse, but on balance think worth raising.
Relevant disclosure: I received a regrant from the FTX Future Fund to investigate the legal needs of effective altruist organisations.
Edit: I want to clarify that I don't think that any particular person knew. I still trust all the same community figures I trusted one week ago, other than folks in the FTX business. For each 'High Profile EA' I can think of, I would be very surprised if that person in particular knew. But even if we think there is only a 0.1% chance that any of the most influential, say, 100 EAs knew, then the chance that none of them knew is 0.999^100, which is about 90.4% (assuming we naively treat those as independent events). If we care about the top 1000 most influential EAs, then we could get that 90.4% chance with just a 0.01% chance of failure.
Edit: I think Ryan Carey's comment is further in the right direction than this post (subject to my view that an independent investigation should stick to fact-finding rather than making philosophical/moral calls for EA) plus I've also had other people contact me spitballing ideas that seem sensible. I don't know what the terms of reference of an investigation would be, but it does seem like simply answering "did anybody know" might be the wrong approach. If you have further suggestions for the sorts of things that should be considered, it might be worth dropping those into the comments.
I think a very relevant question is to ask is how come none of the showy self-criticism contests and red-teaming exercises came up with this? A good amount of time and money and energy were put into such things and if the exercises are not in fact uncovering the big problems lurking in the movement then that suggests some issues
If this comment is more about "how could this have been foreseen", then this comment thread may be relevant. I should note that hindsight bias means that it's much easier to look back and assess problems as obvious and predictable ex post, when powerful investment firms and individuals who also had skin in the game also missed this.
TL;DR:
1) There were entries that were relevant (this one also touches on it briefly)
2) They were specifically mentioned
3) There were comments relevant to this. (notably one of these was apparently deleted because it received a lot of downvotes when initially posted)
4) There has been at least two other posts on the forum prior to the contest that engaged with this specifically
My tentative take is that these issues were in fact identified by various members of the community, but there isn't a good way of turning identified issues into constructive actions - the status quo is we just have to trust that organisations have good systems in place for this, and that EA leaders are sufficiently careful and willing to make changes or consider them seriously, such that all the community needs to do is "raise the issue". And I think looking at the systems within the relevant EA orgs or leadership is what investigations or accountability questions going forward should focus on - all individuals are fallible, and we should be looking at how we can build systems in place such that the community doesn't have to just trust that people who have power and who are steering the EA movement will get it right, and that there are ways for the community to hold them accountable to their ideals or stated goals if it appears to, or risks not playing out in practice.
i.e. if there are good processes and systems in place and documentation of these processes and decisions, it's more acceptable (because other organisations that probably have a very good due diligence process also missed it). But if there weren't good processes, or if these decisions weren't a careful + intentional decision, then that's comparatively more concerning, especially in context of specific criticisms that have been raised,[1] or previous precedent. For example, I'd be especially curious about the events surrounding Ben Delo,[2] and processes that were implemented in response. I'd be curious about whether there are people in EA orgs involved in steering who keep track of potential risks and early warning signs to the EA movement, in the same way the EA community advocates for in the case of pandemics, AI, or even general ways of finding opportunities for impact. For example, SBF, who is listed as a EtG success story on 80k hours, has publicly stated he's willing to go 5x over the Kelly bet, and described yield farming in a way that Matt Levine interpreted as a Ponzi. Again, I'm personally less interested in the object level decision (e.g. whether or not we agree with SBF's Kelly bet comments as serious, or whether Levine's interpretation as appropriate), but more about what the process was, how this was considered at the time with the information they had etc. I'd also be curious about the documentation of any SBF related concerns that were raised by the community, if any, and how these concerns were managed and considered (as opposed to critiquing the final outcome).
Outside of due diligence and ways to facilitate whistleblowers, decision-making processes around the steering of the EA movement is crucial as well. When decisions are made by orgs that bring clear benefits to one part of the EA community while bringing clear risks that are shared across wider parts of the EA community,[3] it would probably be of value to look at how these decisions were made and what tradeoffs were considered at the time of the decision. Going forward, thinking about how to either diversify those risks, or make decision-making more inclusive of a wider range stakeholders[4], keeping in mind the best interests of the EA movement as a whole.
(this is something I'm considering working on in a personal capacity along with the OP of this post, as well as some others - details to come, but feel free to DM me if you have any thoughts on this. It appears that CEA is also already considering this)
If this comment is about "are these red-teaming contests in fact valuable for the money and time put into it, if it misses problems like this"
I think my view here (speaking only for the red-teaming contest) is that even if this specific contest was framed in a way that it missed these classes of issues, the value of the very top submissions[5] may still have made the efforts worthwhile. The potential value of a different framing was mentioned by another panelist. If it's the case that red-teaming contests are systematically missing this class of issues regardless of framing, then I agree that would be pretty useful to know, but I don't have a good sense of how we would try to investigate this.
This tweet seems to have aged particularly well. Despite supportive comments from high-profile EAs on the original forum post, the author seemed disappointed that nothing came of it in that direction. Again, without getting into the object level discussion of the claims of the original paper, it's still worth asking questions around the processes. If there was were actions planned, what did these look like? If not, was that because of a disagreement over the suggested changes, or the extent that it was an issue at all? How were these decisions made, and what was considered?
Apparently a previous EA-aligned billionaire ?donor who got rich by starting a crypto trading firm, who pleaded guilty to violating the bank secrecy act
Even before this, I had heard from a primary source in a major mainstream global health organisation that there were staff who wanted to distance themselves from EA because of misunderstandings around longtermism.
This doesn't have to be a lengthy deliberative consensus-building project, but it should at least include internal comms across different EA stakeholders to allow discussions of risks and potential mitigation strategies.
e.g. A critical review of GiveWell's 2022 cost-effectiveness model, Methods for improving uncertainty analysis in EA cost-effectiveness models, and
Biological Anchors external review
...I for one didn't take the self-criticism stuff particularly seriously, or consider EA to have scored any points by running the contest? And I thought that seemed too obvious to mention, I guess, that of course scoring diligent self-criticism points is harder than that? Zvi had a longer writeup on a similar take.
I have my own sense of what EA is doing all wrong, but it didn't particularly occur to me to try to write it up for the criticism contest. That was obviously going to be an in-frame thing, and was obviously not going to be able to reward anything out-of-frame; doing that is really hard if you don't want to just completely throw the contest over to wild randos. Like, the reason MIRI doesn't run a contest like that is not that our underlying reality contains nothing worth criticizing, but that a contest like that obviously does not work to highlight your real actual problems.
There's nothing particularly unhealthy about a movement where a contest like that fails to turn up any real issues - the contest is obviously doomed in that sense from the start. The only failure is the naivety required to imagine that a contest like that could possibly work, against a semiefficient background where smart people have already been saying various things and there are already reasons why those critiques have not been accepted. The best you can say about a contest like that is that maybe it's worth $100K to get better centralized writeups of in-frame criticisms that your community already knows how to accept but hasn't finished acting on yet.
Have you written that up anywhere? Would be interesting to read.
In hindsight they look a bit performative.
There's a related problem with CEA. Their mission is to nurture the EA community, and their sizeable community health team's is to "preserve the EA community's ability to grow and produce value". But the CH team has focused on public relations, diversity, "discussion norms", and crisis-response, while the community became unhealthy for a different reason - FTX. It seems like this team, or some other, needs to take on a wider mandate, that includes monitoring of such issues.
I would like to know, for example, whether any large EA orgs (and EA thinkers, I suppose too) have formally engaged with those criticism contest entries. Like, was it somebody's job to think about when anything should change as a result? Or something discussed by a governance board (or similar)?