Rereading your post, it does make sense now that you were thinking of safety teams at the big labs, but both the title about "selling out" and point #3 about "capabilities people" versus "safety people" made me think you had working on capabilities in mind.

If you think it's "fair game to tell them that you think their approach will be ineffective or that they should consider switching to a role at another organization to avoid causing accidental harm," then I'm confused about the framing of the post as being "please don't criticize EAs who 'sell out'," since this seems like "criticizing" to me. It also seems important to sometimes do this even when unsolicited, contra point 2. If the point is to avoid alienating people by making them feel attacked, then I agree, but the norms proposed here go a lot further than that.

(Realizing that it would be hypocritical for me not to say this, so I'll add: if you're working on capabilities at an AGI lab, I do think you're probably making us less safe and could do a lot of good by switching to, well, nearly anything else, but especially safety research.)

Setting aside the questions of the impacts of working at these companies, it seems to me like this post prioritizes the warmth and collegiality of the EA community over the effects that our actions could have on the entire rest of the planet in a way that makes me feel pretty nervous. If we're trying in good faith to do the most good, and someone takes a job we think is harmful, it seems like the question should be "how can I express my beliefs in a way that is likely to be heard, to find truth, and not to alienate the person?" rather than "is it polite to express these beliefs at all?" It seems like at least the first two reasons listed would also imply that we shouldn't criticize people in really obviously harmful jobs like cigarette advertising.

It also seems quite dangerous to avoid passing judgment on individuals within the EA community based on our impressions of their work, which, unless I'm missing something, is what this post implies we should do. Saying we should "be kind and cooperative toward everyone who is trying in good faith to reduce AI risk" kind of misses the point, because a lot of the evidence for them "trying in good faith" comes from our observations of their actions. And, if it seems to me that someone's actions make the world worse, the obvious next step is "see what happens if they're presented with an argument that their actions are making the world worse." If they have responses that make sense to me, they're more likely to be acting in good faith. If they don't, this is a significant red flag that they're not trustworthy, regardless of their inner motivations: either factors besides the social impact of their actions are dominating in a way that makes it hard to trust them, or their judgment is bad in a way that makes it hard to trust them. I don't get this information just by asking them open-ended questions; I get it by telling them what I think, in a polite and safe-feeling way.

I think the norms proposed in this post result in people not passing judgment on the individuals working at FTX, which in turn leads to trusting these individuals and trusting the institution that they run. (Indeed, I'm confused at the post's separation between criticizing the decisions/strategies made by institutions and those made by the individuals who make the decisions and choose to further the strategies.) If people had suspicions that FTX was committing fraud or otherwise acting unethically, confronting individuals at FTX with these suspicions -- and forming judgments of the individuals and of FTX -- could have been incredibly valuable.

Weaving these points together: if you think leading AGI labs are acting recklessly, telling this to individuals who work at these labs (in a socially competent way) and critically evaluating their responses seems like a very important thing to do. Preserving a norm of non-criticism also denies these people the information that (1) you think their actions are net-negative and (2) you and others might be forming judgments of them in light of this. If they are acting in good faith, it seems extremely important that they have this information -- worth the risk of an awkward conversation or hurt feelings, both of which are mitigable with social skills.

Also seems important to note: EA investing should have not only different expectations about the world but also different goals about the point of finance. We are relatively less interested in moving money from worlds where we're rich to worlds where we're poor (the point of risk aversion) and more interested in moving it from worlds where it's less useful to where it's more useful.

Concretely: if it turns out we're wrong about AI, this is a huge negative update on the usefulness of money to buy impact, since influencing this century's development generally becomes much less important. So even if we think the stocks are accurately priced, we probably want to bet a lot of money on the market being surprised in the direction of AI being huge and timelines being short, because if those are true, money would be extremely useful, and it's potentially worth giving up a lot of money in worlds where it's false.

(Edited to add: even more concretely, this means that stocks correlated to AI progress are underpriced for us even if they're not underpriced for a typical investor, and in particular riskier assets correlated to AI progress like options on AI/semiconductor/datacenter stocks are likely especially underpriced for our preferences. Also n.b. by "our preferences" I mean as donors -- I'm talking about the overall longtermist portfolio, not making investment advice for any individuals.)

Agree with Xavier's comment that people should consider reversing the advice, but generally confused/worried that this post is getting downvoted (13 karma on 18 votes as of this writing). In general, I want the forum to be a place where bold, truth-seeking claims about how to do more good get attention. My model of people downvoting this is that they are worried that this will make people work harder despite this being suboptimal. I think that people can make these evaluations well for themselves, and that it's good to present people with information and arguments that might change their mind. Just as "donate lots of your money to global charities" and "stop buying animal products" are unwelcome to hear and might be bad for you if you take them to extremes in your context but are probably good for the world, "consider working more hours" could be bad in some cases but also might help people learn faster and become excellent at impactful work, and we should be at least be comfortable debating whether we're at the right point on the curve. 

Heavily cosigned (as someone who has worked with some of Nick's friends whom he got into EA, not as someone who's done a particularly great job of this myself). I encourage readers of this post to think of EA-sympathetic and/or very talented friends of theirs and find a time to chat about how they could get involved!

Wait, why have we not tried to buy ea.org?

As your fellow Cantabrigian I have some sympathies for this  argument. But I'm confused about some parts of it and disagree with others:

• "EA hub should be on the east coast" is one kind of claim. "People starting new EA projects, orgs, and events should do so on the east coast" is a different one. They'd be giving up the very valuable benefits of living near the densest concentrations of other orgs, especially funders. You're right that the reasons for Oxford and the Bay being the two hubs are largely historical rather than practical, but that's the nature of Schelling points; it might have been better to have started in the East Coast (or somewhere temperate, cheap, cosmopolitan, and globally centrally located like Barcelona), but how are we going to all coordinate to move there? The options that come to mind (Open Phil, FTX, CEA, and/or others move there, or coordinate to do so together?) seem very costly — on the order of weeks or months of the entire organization's time.
• By the commonly held view that AI is by far the most important cause area, it's fine that the Bay is an EA hub despite the tech industry being its only non-Schelling-point reason to be a hub.
• For better or worse, Berkeley is also a hub for community-building now; tons of student organizers spent this summer there. Again, they go there for the recursive common-knowledge reason that other people will also be going there, so there'd have to be some (costly?) coordinated shift probably driven by a major org.
• Seems slightly like cheating to count all those universities (or indeed all those cities) as part of the same hub. Oxford and London are way closer than any of Boston, DC, and NYC are to each other. It seems like a place can be a hub if it would be physically easy for any two people living in it to meet every week. Boston, NYC, and DC are not close enough to qualify. Pointing out the cause area networks that each of these cities have, and cumulatively counting them against the Bay "merely" having the AI industry, makes it seem more likely than it is that the entire East Coast could achieve the kind of Schelling status that Berkeley has. (Indeed, notably the Bay Area EA community is overwhelmingly located specifically in Berkeley, supporting the idea that physical proximity is very important.)
• Generally I really like the East Coast lifestyle (insofar as it differs from the Bay's) and am figuring out how to articulate it. Maybe it's that people are a little more ironic. Maybe it's that having to Uber basically everywhere in the Bay is dystopian. That being said, lots of EAs like the outdoors, and the East Coast is much worse than the Bay Area for hiking etc.
• One thing that I like about Boston relative to the Bay is the relatively horizontal social/professional structure: it feels like, in the Bay, there's a pretty clear status pyramid and a pretty clear line of who's in the elite circle (access to the top workspaces), while it's looser and chiller in Boston. But it seems like this results from the Bay being a major hub and Boston being less of a hub. E.g., once a certain office space opens in Cambridge, I expect some of these dynamics to reappear, and if Boston became as booming as Berkeley, I think a pyramid would likely start to become more apparent as well. (Sad.)

I think it is a heuristic rather than a pure value. My point in my conversation with Josh was to disentangle these two things — see Footnote 1! I probably should be more clear that these examples are Move 1 in a two-move case for longtermism: first, show that the normative "don't care about future people" thing leads to conclusions you wouldn't endorse, then argue about the empirical disagreement about our ability to benefit future people that actually lies at the heart of the issue.

Borrowing this from some 80k episode of yore, but it seems like another big (but surmountable) problem with neglectedness is which resources count as going towards the problem. Is existential risk from climate change neglected? At first blush, no, hundreds of billions of dollars are going toward climate every year. But how many of these are actually going towards the tail risks, and how much should we downweight them for ineffectiveness, and so on.