Seems wrong to me. For example, Hamilton and Jefferson on the U.S. patent system. Also debates between Tesla and Edison on DC vs. AC and various policies associated with electricity. Also people working on nuclear regulation. Also people working on the early internet. Also lots of stuff around genetic engineering, the trajectory of which seems to have been shaped by many people taking deliberate action. Also, climate change, where a bunch of people did successfully invest in solar stuff which seems like it's now paying off quite well, and certainly didn't seem that obvious.

I don't know whether any of these count as "non-obvious". I feel like the "obvious" is doing all the work here, or something.

Sure, you can slice things here however you want. I do think it's important to be at least consistent with both positives and negatives here. I've definitely seen people claim credit for GiveWell as part of EA, which of course was also founded pre-2011. I don't think think there is a clear answer on how to handle this, and it seems most important to just be consistent.

IMO it does also feel really weird to say the sentence "we've had a substantial influence on 2 out of the 3 top AI capability companies", when like, the same people also had a substantial influence on the third. 

Cool, makes sense. Seems like we are mostly on the same page on this subpoint.

I'm just saying that the argument "this is a suicide race" is really not the way we should go. We should say the risk is >10% and that's obviously unacceptable, because that's an argument we can actually win.

Hmm, just to be clear, I think saying that "this deployment has a 1% chance of causing an existential risk, so you can't deploy it" seems like a pretty reasonable ask to me. 

I agree that I would like to focus on the >10% case first, but I also don't want to set wrong expectations that I think it's reasonable at 1% or below. 

I agree that AI developers should have to prove that the systems they build are reasonably safe. I don't think 100% is a reasonable ask, but 90% or 99% seem pretty safe.

Sorry, just to clarify, what do we mean by "safe" here? Clearly a 90% chance of the system not disempowering all of humanity is not sufficient (and neither would a 99% chance, though that's maybe a bit more debatable), so presumably you mean something else here. 

Shane Legg started DeepMind together with Demis after he heard about about Superintelligence concerns by Eliezer and Kurzweil, and was active for many months on LessWrong reading Eliezer's arguments and commenting on the site. 

See this tweet thread: https://twitter.com/ShaneLegg/status/1598047654159978496 

And this old LessWrong account from Shane: https://www.lesswrong.com/users/shane_legg 

Just to be clear, under most ethical systems this is a lower bound.

Humanity going extinct is a lot worse than 8 billion people dying, unless you don't care at all about future lives (and you don't care about the long term goals of present humans, most of which have at least some goals that extend beyond their death).

Hmm, yeah, I do think there is a countersignaling thing going on a bit with those terms, though I do find it hard to forecast whether that will help or harm the term overall. The modern internet and public opinion is a really weird place. Like, take the term MAGA, which sure took off and was really successful and feels quite similar to "foom".

I am not really sure what this post is arguing against, or who it is arguing against. I feel like it just kind of implies that "cute pet name"s are low-status and therefore memetically unviable, which seems dubious to me (especially in the internet age people use all kinds of cute abbreviations).

I haven't seen anyone actively advocate for using "foom" and I've most recently only seen it used by people not actually very central to the EA, Rationality or AI Alignment communities. I don't think anyone is "trying to make it happen", it's just like the current most natural term we have for it. I don't think anyone is super attached to it, and you can just propose a different name and then people might go with that one, or they might not, forecasting the adoption of terms is quite hard and I haven't seen people have a lot of success with it.