It's difficult if the format requires a 1D sliding scale. I think reasonable positions can be opposed on AI vs other GCRs vs infrastructure vs evidenced interventions, and future (if it exists) is default bad vs future is default good, and perhaps 'future generations should be morally discounted' vs not.

I'm going to struggle to cast a meaningful vote on this, since I find 'existential risk' terminology as used in the OP more confusing than helpful, since e.g. it includes nonexistential considerations and in practice excludes non-extinction catastrophes from a discussion they should very much be in, in favour of work on the heuristical-but-insufficient grounds of focusing on events that have maximal extinction probability (i.e. AI). 

I've argued here that non-extinction catastrophes could be as or more valuable to work on than immediate extinction events, even if all we care about is the probability of very long-term survival. For this reason I actually find Scott's linked post extremely misleading, since it frames his priorities as 'existential' risk, then pushes people entirely towards working on extinction risk - and gives reasons that would apply as well to non-extinction GCRs. I gave some alternate terminology here, and while I don't want to insist on my own clunky suggestions, I wish serious discussions would be more precise.

Do you have examples of LLMs improving Fermi estimates? I've found it hard to get any kind of credences at all out of them, let alone convincing ones.

Yup! At least, I make a point of coworking most of the working day (albeit in Singapore time) and often at least being on in the evenings. There are various semiregular users, with probably the biggest density (albeit some variance) in the European working day.

For what it's worth, there used to be an 80k pledge along similar lines. They quietly dropped it several years ago, so you might want to find someone involved in that decision to try and understand why (I suspect and dimly remember that it was some combination of non-concreteness, and concerns about other-altruism-reduction effects).

I'm confused how you square the idea of 'an internal and consistent self-identity that remains intact in an AI over time (similar to what exists in humans)' with your advocacy for eliminativism about consciousness. What phenomenon is it you think is internal to humans?

Do you have a sense of what proportion of biomass they made up from 1900-1950?

I happen to strongly agree that moral discount rate should be 0, but a) it's still worth acknowledging that as an assumption, and b) I think it's easy for both sides to equivocate it with risk-based discounting. It seems like you're de facto doing when you say 'Under that assumption, the work done is indeed very different in what it accomplishes' - this is only true if risk-based discounting is also very low. See e.g. Thorstad's Existential Risk Pessimism and the Time of Perils and Mistakes in the Moral Mathematics of Existential Risk for formalisms of why it might not be - I don't agree with his dismissal of a time of perils, but I do agree that the presumption that explicitly longtermist work is actually better for the long term than short-to-medium-term focused work is is based on little more than Pascalian handwaving.

I'm confused by your paragraph about insurance. To clarify:

• I don't expect insurance companies to protect against either extinction catastrophes or collapse-of-civilisation catastrophes, since as you say such catastrophes are uninsurable.
• I suspect they also don't protect against medium-damage-to-civilisation catastrophes for much the same reason - I don't think insurance has the capacity to handle more than very mild civilisational shocks.
• I do think government organisations, NGOs and academics have done very important work in the context of reducing risks of civilisation-harming events.
• I think that if you assign a high risk of a post-catastrophe civilisation struggling to flourish (as I do), these events look comparably as bad from a long-term perspective as extinction once you also account for their greater likelihood. I suggested a framework for this analysis here and built some tools to implement it described here.

Of course you can disagree about the high risk to flourishing from non-existential catastrophes but that's going to be a speculative argument about which people might reasonably differ. To my knowledge, no-one's made the positive case in depth, and the few people who've looked seriously into our post-catastrophe prospects seem to be substantially more pessimistic than those who haven't. See e.g.:

• Dartnell - 'we might have scuppered the chances of any society to follow in our wake'
• Rodriguez - 'humanity might be stagnant for millennia'
• Jebari - 'the development of industrialisation depends on more factors, and is more of a "lucky shot", than we might otherwise think'

Thanks - agree or disagree with it, this is a really nice example of what I was hoping for.

The extent to which you think they're the same is going to depend heavily on 

1. your long term moral discounting rate (if it's high, then you're going to be equally concerned between highly destructive events that very likely won't kill everyone and comparably destructive events that might),
2. your priors on specific events leading to human extinction (which, given the lack of data, will have a strong impact on your conclusion), and
3. your change in credence of civilisation flourishing post-catastrophe.

Given the high uncertainty behind each of those considerations (arguably excluding the first), I think it's too strong to say they're 'not the same at all'. I don't know what you mean by fields only looking into regional disasters - how are you differentiating those investigations from the fields that you mention that the general public has heard of in large part because a ton of academic and governmental effort has gone into it?