Scott Alexander introduces the ‘noncentral fallacy’ as follows: “X is in a category whose archetypal member gives us a certain emotional reaction. Therefore, we should apply that emotional reaction to X, even though it is not a central category member."

This post seems like an archetypal instance of the noncentral fallacy.

Intuitively, it seems we should respond differently depending on which of these three possibilities is true:

1. They think that what they are doing is negative for the world, but do it anyway, because it is good for themselves personally.
2. They do not think that what they are doing is negative for the world, but they believe this due to motivated cognition.
3. They do not think that what they are doing is negative for the world, and this belief was not formed in a way that seems suspect.

From an act consequentialist perspective, these differences do not matter intrinsically, but they are still instrumentally relevant.^[1]

1. ^{^}

   I don't mean to suggest that any one of these possibilities is particularly likely, or they they are all plausible. I haven't followed this incident closely. FWIW, my vague sense is that the Mechanize founders had all expressed skepticism about the standard AI safety arguments for a while, in a way that seems hard to reconcile with (1) or (2).

By the way, the name is ‘Jaime’, not ‘Jamie’. The latter doesn't exist in Spanish and the two are pronounced completely differently (they share one phoneme out of five, when aligned phoneme by phoneme).

(I thought I should mention it since the two names often look indistinguishable in written form to people who are not aware that they differ.)

Thanks for the clarification. I didn’t mean to be pedantic: I think these discussions are often unclear about the relevant time horizon. Even Bostrom admits (somewhere) that his earlier writing about existential risk left the timeframe unspecified (vaguely talking about "premature" extinction).

On the substantive question, I’m interested in learning more about your reasoning. To me, it seems much more likely that Earth-originating intelligence will go extinct this century than, say, in the 8973th century AD (conditional on survival up to that century). This is because it seems plausible that humanity (or its descendants) will soon develop technology with enough destructive potential to actually kill all intelligence. Then the question becomes whether they will also successfully develop the technology to protect intelligence from being so destroyed. But I don’t think there are decisive arguments for expecting the offense-defense balance to favor either defense or offense (the strongest argument for pessimism, in my view, is stated in the first paragraph of this book review). Do you deny that this technology will be developed "over time horizons that are brief by cosmological standards”? Or are you confident that our capacity to destroy will be outpaced by our capacity to prevent from destruction?

The extinction of all Earth-originating intelligent life (including AIs) seems extremely unlikely.

Extremely unlikely to happen... when? Surely all Earth-originating intelligent life will eventually go extinct, because the universe’s resources are finite.

Here's another summary. I used Gemini 2.0 Flash (via the API), and this prompt:

The following is a series of comments by Habryka, in which he makes a bunch of criticisms of the effective altruism (EA) movement. Please look at these comments and provide a summary of Habryka’s main criticisms.

1. Lack of leadership and accountability: He believes EA leadership is causing harm and lacks mechanisms for correcting course.
2. Emphasis on PR and narrative control: He condemns EA organizations' risk aversion, guardedness, and attempts to control the narrative around FTX, prioritizing public image over transparency.
3. Inadequate community health: He laments conformity pressures, fears of reprisal for dissent, and insufficient efforts to cultivate a culture of open disagreement.
4. Entanglement with FTX: He faults EA leadership, particularly Will MacAskill, for endorsing Sam Bankman-Fried and entangling the movement with FTX despite warnings about SBF's character.
5. Hero worship and lack of respect for intellectual leaders: He criticizes the hero worship of MacAskill, contrasting it with MacAskill's perceived lack of engagement with other intellectual leaders in the community. He sees this as part of a pattern of MacAskill prioritizing popularity and prestige over community health and epistemic integrity.
6. Misleading communications and lack of transparency: He criticizes CEA for making inaccurate and misleading statements, for omitting crucial context in communications, and for concealing information about funding decisions.
7. Scaling too quickly and attracting grifters: He worries that EA's rapid growth and increased funding attract deceptive actors and create perverse incentives.
8. Overreliance on potentially compromised institutions: He expresses concerns about EA's deep ties to institutions like Oxford University, which may stifle intellectual exploration and operational capacity.
9. Ignoring internal warnings about FTX: He reveals that he and others warned EA leadership about Sam Bankman-Fried's reputation for dishonesty, but those warnings went unheeded. He suggests he personally observed potentially illegal activities by SBF but chose not to share this information more widely.
10. Flawed due diligence and poor judgment in grantmaking: He feels EA leadership's due diligence on SBF was inadequate and that they made poor judgments in providing him with substantial resources. He extends this criticism to grantmaking practices more generally.
11. Unfair distribution of resources: He argues that the current distribution of funds within EA doesn't adequately compensate those doing object-level work and undervalues their contributions relative to donors. He argues for a system that recognizes the implicit tradeoff many have made in pursuing lower-paying EA-aligned careers.
12. Centralized media policy and negative experiences with journalists: While supporting a less centralized media policy, he also cautions against interacting with journalists, as they frequently misrepresent interviewees and create negative experiences.
     

New episode:

Laura Sofía Castro sobre la generación y uso de evidencia en políticas públicas y medición de impacto
 

80k has made important contributions to our thinking about career choice, as seen e.g. in their work on replaceability, career capital, personal fit, and the ITN framework. This work does not assume a position on the neartermism vs. longtermism debate, so I think the author’s neartermist sympathies can’t fully explain or justify the omission.

It may be also worth noting that the website is available in German and Spanish.