Junior EAs getting advice that helps them grow: "I'm getting reard on"

So you're saying he reard you?

My sense is that there’s been a significant shift in how much longtermists prioritise non-extinction risks over the last few years. A decade ago people who were trying to ensure the flourishing of the universe trillions of years from now were very often focused on avoiding events that would kill all humans.

I agree that this shift has happened over the past 2 years. But I think 2 to 3 years ago EAs were unusually focused on extinction compared to a decade ago. I remember more discussions back then around positive visions for the longterm.

For what it’s worth, we just announced our first Frontier Biodefense Fellowship at Pivotal, which is more singularly focused on avoiding extinction than most projects within AI safety (including our AI safety fellowship). Obviously the team has a range of motivation to work on Biodefense, but for me weak longtermist arguments are quite central.

What's your explanation for why they attack EAs rather than, say, the AI ethics crowd?

I think there are a few plausible reasons that don't require "undemocratic power-seeking" as the primary explanation:

• EAs have been more motivated to & competent at gaining influence through legitimate/standard means
• EA policy ideas enjoyed better reception by policymakers, were more aligned with their interests. The opponents might have stronger disagreement with AI ethics ideas, but see them as much less likely to have influence.
• EAs compete more directly on their territory. The AI ethics crowd has sufficiently different values and assumptions that they're less of a threat locally. (Kind of a 'narcissism of small differences'.)

I expect that if your ideas are resonating with policymakers and people are getting appointed to relevant roles because they're competent, bad faith opponents will target you roughly the same as if you'd been pulling strings behind the scenes in dubious ways.

Why was SB 1047 so controversial, while other much more onerous AI bills (esp for "little tech") were barely discussed?

Maybe I'm missing something. SB 1047 seemed like a relatively transparent action, that followed the democratic process. Is your point that undemocratic power-seeking actions prior/unrelated to SB 1047 likely explains the stronger opposition to SB 1047?

I haven't done philosophy in a while, might be missing something, but wanted to highlight what I think is the strongest objections to the view^[1] in a way that may be more salient than the framing in section 6. It's probably a reason why many might prefer a total view.

To be clear, I do think the Saturation View improves on other non-total views I know of, and I appreciate that they flag some of its hard-to-stomach implications. But I still think the post understates how bad the separability issue is. So here are two short points:

Non-separability is really bad.

The core problem is that facts about/experiences of wholly unaffected people can change the value of the affected person's experiences. If there are already sufficiently many people elsewhere with sufficiently similar experiences, then an additional person having an extremely deep, meaningful, happy life adds near-zero marginal value. That seems very hard to accept.

And for negative experiences the implication is potentially even less intuitive. An additional torturous experience can add almost no marginal disvalue if enough sufficiently similar torture already exists. They discuss this under the “cheap suffering” problem & call it the strongest argument against the view, but I think it is worth emphasizing just how unintuitive of a conclusion this is. From the victim’s perspective, the torture is not any less bad because other similar torture already occurred. But the saturation view says that, from the point of view of population value, their torturous experience would matter hardyl at all.

ETA: Relatedly, the view assigns value to our experiences depending on empirically inaccessible facts. Whether sufficiently distant aliens have sufficiently similar experiences is something we probably can't know, but it would radically change how our actions matter. That seems strange.

I don't think the 'tameness' of the view recovers that much?

My understanding is that the Saturation View does better because violations of separability are localized. Ancient Egyptians or distant aliens only affect the marginal value of new lives if their experiences are sufficiently similar. So in many "normal situations", the view behaves roughly separably.

But the separability worry still holds with sufficiently large numbers. If enough sufficiently similar unaffected lives exist elsewhere, they can radically change the marginal value of what we do here.

And population ethics is full of large-number objections. The Repugnant Conclusion itself gets its core intuitive force from considering sufficiently enormous populations, and is also not a “normal situation.” So if the Saturation View is partly motivated by avoiding the very bad large-number implications of total views, then its own large-number implications seem fair game too.

1. ^{^}

   the authors agree with this, afaict

I’m sad to announce that I’m leaving academia.

I’m looking forward to working on AI safety.

Good point, they might know. Does anyone know a neurotypical? Or a friend of a neurotypical that I could reach out to?

Attacking people's outfit choices is a unique low for the EA forum!

THE EA FORUM TEAM COULD REALLY HELP HERE BY ADDING A 'HEADING 0' AND 'HEADING -1' FORMATTING OPTION.

Thanks for writing this!

You're describing integral altruism as broader than EA, but if I understand you correctly, it's also narrower in many ways. Some examples:

Letting go of the need to control everything and transcending the frame that we are in conflict with the natural unfolding of the universe. This also means emphasising collective action over individual heroism.

–> Effective altruism doesn't take a position on whether we are in conflict with the natural unfolding of the universe. EAs emphasise collective actions vs. individual heroism to various degrees.

take radical uncertainty seriously

–> EAs already do this to various degrees. If integral altruists take this really seriously, they are a subset of EAs in this regard.

altruism grounded in truth rather than being driven by guilt or pride

–> EA doesn't say where your altruistic motivation should be grounded in. All of the reasons you list are considered viable (although people of course disagree to what degree they are conducive/to be encouraged).

Some of the things you describe (especially the 'different ways of knowing') seem to sit more outside of what is common within EA. There it seems more like integral altruism actually is broader.

Overall I'm not completely sure whether integral altruism is a way of doing effective altruism differently, or a competing (though often overlapping) world view.