The next existential catastrophe is likelier than not to wipe off all animal sentience from the planet

Intuitively seems very unlikely.

1. The Chicxulub impact wiped out dinosaurs but not smaller mammals, fish, and insects. Even if a future extinction event caused a total ecosystem collapse, I would expect that some arthropods will  be able to adapt and survive.
2. I feel a goal-driven, autonomous ASI won't care much about the majority of non-humans. We don't care about anthills we trample when constructing buildings (ideally, we should); similarly an ASI would not intentionally target most non-humans — they aren't competing for the same resources or obstructing the ASI's goals.

Thanks, great post! 

A few follow-up questions and pushbacks:

• Even if cannibalization happens, here are three questions that "multiple well-designed studies analyzing substitution effects demonstrated that cultivated and plant-based meats appeal to sufficiently different consumer segments" may not answer:
   
  • Would commercially viable cultivated meat more favorably alter consumer preferences over time?
  • A non-negligible portion of veg*ns abandon their veg*nism — would introduction of cultivated meat improve retention of animal-free consumption patterns?

  • How would introduction of cultivated meat affect flexitarian dietary choices? Flexitarians eat a combination of animal- and plant-based meat. When cultivated meat becomes commercially viable, would flexitarians replace the former or the latter with cultivated meat?

    If the answer is a yes to any of these, I think that is a point in favor of cultivated meat. I expect cultural change to be a significant driver of reduced animal consumption, and this cultural change will only be possible if there is a stable class of consumers who normalize consumption of animal-free products.

To draw a historical parallel, when industrial chicken farming developed in the second half of the 20th century, people didn't eat less of other meats; they just ate chicken in addition.

• 

  Is this true? It seems that as chicken did displace beef consumption by 40% (assuming consumption ~ supply) or am I grossly misunderstanding the chart above?

  • Further, isn't there an upper bound to how much addition can happen? Meat became cheap and widely available, incomes rose, people started eating more of everything, so consumption increased. But there is only so much more that one can eat, so at some point people started making cost-based trade-offs between beef and chicken. If cultivated chicken were to be cheaper than animal-based beef or chicken, shouldn't we expect people to start making similar trade-offs?

AGI by 2028 is more likely than not

I hope to write about this at length once school ends, but in short, here are the two core reasons I feel AGI in three years is quite implausible:
 

• The models aren't generalizing. LLM's are not stochastic parrots, they are able to learn, but the learning heuristics they adopt seem to be random or imperfect. And no, I don't think METR's newest is evidence against this.^[1]
   
• It is unclear if models are situationally aware, and currently, it seems more likely than not that they do not possess this capability. Laine et al. (2024) shows that current models are far below human baselines of situational awareness when tested on MCQ-like questions. I am unsure how models would be able to perform long-term planning—a capability I would consider is crucial for AGI—without being sufficiently situationally aware.

1. ^{^}

   As Beth Barnes put it, their latest benchmark specifically shows that "there's an exponential trend with doubling time between ~2 -12 months on automatically-scoreable, relatively clean + green-field software tasks from a few distributions." Real world tasks rarely have such clean feedback loops; see Section 6 of METR's RE-bench paper for a thorough list of drawbacks and limitations.

Should EA avoid using AI art for non-research purposes?

Voting under the assumption that by EA, you mean individuals who are into EA or consider themselves to be a part of the movement (see "EA" is too vague: let's be more specific).

Briefly, I think the market/job displacement and environmental concerns are quite weak, although I think EA professionals should avoid using AI art unless necessary due to reputational and aesthetic concerns. However, for images generated in a non-professional context, I do not think avoidance is warranted.

(meta: why are people downvoting this comment? I disagree voted but there is nothing in this comment that makes me go, "I want less comments like this on the Forum")

This helps. That is not at all how I interpreted 'our answer to both of your questions is "no."' Apologies!

it's also a big clear gap now on the trusted, well known non-AI career advice front

From the update, it seems that:

• 80K's career guide will remain unchanged
  • I especially feel good about this, because the guide does a really good job of emphasizing the many approaches of pursuing an impactful career
  • n = 1 anecdotal point: during tabling early this semester, a passerby mentioned that they knew about 80K because a professor had prescribed one of the readings from the career guide in their course. The professor in question and the class they were teaching had no connection with EA, AI Safety, or our local EA group.
    • If non-EAs also find 80K's career guide useful, that is a strong signal that it is well-written, practical, and not biased to any particular cause
    • I expect and hope that this remains unchanged, because we prescribe most of the career readings from that guide in our introductory program
• Existing write-ups on non-AI problem profiles will also remain unchanged
• There will be a separate AGI career guide
• But the job board will be more AI focused

Overall, this tells me that groups should still feel comfortable sharing readings from the career guide and on other problem profiles, but selectively recommend the job board primarily to those interested in "making AI go well" or mid/senior non-AI people. Probably Good has compiled a list of impact-focused job boards here, so this resource could be highlighted more often.

1. Another place people could be directed for career advice: https://probablygood.org/

2. Since last semester, we have made career 1-on-1s a mandatory part of our introductory program.

   1. This semester, we will have two 1-on-1s
      1. The first one will be a casual conversation where the mentee-mentor get to learn more about each other
      2. The second one will be more in-depth, where we will share this 1-on-1 sheet (shamelessly poached from the 80K), the mentees will fill it out before the meeting, have a ≤1 hour long conversation with a mentor of their choice, and post-meeting, the mentor will add further resources to the sheet that may be helpful.

   The advice we give during these sessions ends up being broader than just the top EA ones, although we are most helpful in cases where:

   — someone is curious about EA/adjacent causes
   — someone needs graduate school related questions
   — general "how to best navigate college, plan for internships, etc" advice
   
   Do y'all have something similar set up? 

Makes sense. Just want to flag that tensions like these emerge because 80K is simultaneously a core part of the movement and also an independent organization with its goals and priorities.