Head of Lightcone Infrastructure. Wrote the forum software that the EA Forum is based on. Often helping the EA Forum with various issues with the forum. If something is broken on the site, it's a good chance it's my fault (Sorry!).
A thing you might not know is that I was on the founding team of the EA Global series (and was in charge of EA Global for roughly the first two years of its existence). This of course doesn't mean I am right in my analysis here, but it does mean that I have a lot of detailed knowledge about the kind of community negotiations that were going on at the time.
I agree with a bunch of the arguments you made, but my sense is that when creating EA Global, CEA leaned heavily on its coordinating role within the community (which I think made sense).
Indeed, CEA took over the EA Summit from Leverage explicitly because both parties thought it was pretty important to have a centralized annual EA conference.
Huh, EAG feels like one of the most obvious community-institutions. Like, it's the central in-person gathering event of the EA community, and it's exactly the kind of thing where you want to empower an organization to run a centrally controlled version of it, because having a Schelling-event is very valuable.
But of course, in empowering someone to do that, CEA accepts some substantial responsibility to organize the event with the preferences of the community in mind. Like, EAG is really hard to organize if you are not in an "official EA-representative" position, and a huge fraction of the complexity comes from managing that representation.
I am not AGB, but it's clear that a huge fraction of the power that CEA has comes from it being perceived as a representative of the EA community, and because the community empowered it to solve coordination problems between its members. That power is given conditional on CEA acting on behalf of the people who invested that power.
Sure, maybe CEA accepted those resources (and the expectations that came with that) with the goal of doing the most good, but de-facto CEA as an institution basically only exists because of its endorsement by the EA community, and the post as written seems to me like it basically is denying that power relationship and responsibility.
Lightcone in its stewardship of LW is in a very similar position. Our goal with LW is to develop an art of rationality and reduce existential risk, but as an institution we are definitely also responsible for optimizing for the goals of the other stakeholders who have invested in LessWrong (like the authors, commenters, Eliezer who founded the site, and the broader rationality community which has invested in LessWrong as a kind of town square). People would be really pissed if we banned long-term contributors to LW, even if we thought it was best by our own lights, and rightfully so. They have invested resources which make them a legitimate stakeholder in the commons that we are administering.
(there is some degree to which we do have leeway here because there is widespread buy-in for something like "Well Kept Gardens Die by Pacifism", but that leeway comes from the fact that there is widespread buy-in for discretion-based moderation, and that buy-in does not exist for all forms of possible changes to LW)
This is an interesting datapoint, though... just to be clear, I would not consider the Manhattan project a success on the dimension of wisdom or even positive impact.
They did sure build some powerful technology, and they also sure didn't seem to think much about whether it was good to build that powerful technology (with many of them regretting it later).
I feel like the argument of "the only other community that was working on technology of world-ending proportions, which to be clear, did end up mostly just running full steam ahead at building the world-destroyer, was also very young" is not an amazing argument against criticism of EA/AI-Safety.
I think "5%" is just very badly defined. If I just go with the most intuitive definition to me, then 32.5 good video explainers would probably improve the AI x-risk relevant competence of the US government by more than 5% (which currently is very close to 0, and 5% of a very small number is easy to achieve).
But like, any level of clarification would probably wildly swing whatever estimates I give you. Disagreement on this question seems like it will inevitably just lead to arguing over definitions.
If the marginal scholar is better than the median scholar, why would you just not admit the worst scholars and then admit the better scholars? Clearly the marginal scholar would usually be the worst scholar? Are you saying that if you had half the money that the average quality of the cohort would go down instead of up, and that you would be unable to prioritize only admitting the more competent people?
CEA receives many fewer resources from its donors than from the community. Again, CEA would not really have a job without the community. An organization like CEA would totally exist without your big donors (like, the basic institution of having an "EA leadership organization" requires a few hundred k per year, which you would be able to easily fundraise from a very small fraction of the community, and the labor-value of the people who are substantially directing their life based on the broader EA community vastly eclipses the donations to CEA).
Your donors seem obviously much less important of a stakeholder than the community which is investing you with the authority to lead.