I don't see how this engages with the arguments I cited, or the cited post more generally. Why do you think it's plausible to form a (non-arbitrary) determinate judgment about these matters? Why think these determinate judgments are our "best" judgment, when we could instead have imprecise credences that don't narrow things down beyond what we have reason to?

I don't think this response engages with the argument that judgment calls about our impact on net welfare over the whole cosmos are extraordinary claims, so they should be held to a high epistemic standard. What do you think of my points on this here and in this thread?

I think this is the most honest answer, from an impartial altruistic perspective.

I've got a big moral circle (all sentient beings and their descendants), but it does not extend to aliens because of cluelessness. 

...

I'm quite confident that if we're thinking about the moral utility of spacefaring civilisation, we should at least limit our scope to our own civilisation

I agree that the particular guesses we make about aliens will be very speculative/arbitrary. But "we shouldn't take the action recommended by our precise 'best guess' about XYZ" does not imply "we can set the expected contribution of XYZ to the value of our interventions to 0". I think if you buy cluelessness — in particular, the indeterminate beliefs framing on cluelessness — the lesson you should take from Maxime's post is that we simply aren't justified in saying any intervention with effects on x-risk is net-positive or net-negative (w.r.t. total welfare of sentient beings).

This is linked to my discussion with Jim about determinate credences (since I didn’t initially understand this concept well, ChatGPT gave me a useful explanation).

FYI, I don't think ChatGPT's answer here is accurate. I'd recommend this post if you're interested in (in)determinate credences.

To be clear, "preferential gap" in the linked article just means incomplete preferences. The property in question is insensitivity to mild sweetening.

If one was exactly indifferent between 2 outcomes, I believe any improvement/worsening of one of them must make one prefer one of the outcomes over the other

But that's exactly the point — incompleteness is not equivalent to indifference, because when you have an incomplete preference between 2 outcomes it's not the case that a mild improvement/worsening makes you have a strict preference. I don't understand what you think doesn't "make sense in principle" about insensitivity to mild sweetening.

I fully endorse expectational total hedonistic utilitarianism (ETHU) in principle

As in you're 100% certain, and wouldn't put weight on other considerations even as a tiebreaker? That seems extreme. (If, say, you became convinced all your options were incomparable from an ETHU perspective because of cluelessness, you would presumably still all-things-considered-prefer not to do something that injures yourself for no reason.)

Thanks! I'll just respond re: completeness for now.

1. When we ask "why should we maximize EV," we're interested in the reasons for our choices. Recognizing that I'm forced by reality to either donate or not-donate doesn't help me answer whether it's rational to strictly prefer donating, strictly prefer not-donating, be precisely indifferent, or none of the above.
2. Incomplete preferences have at least one qualitatively different property from complete ones, described here, and reality doesn't force you to violate this property.
3. Not that you're claiming this directly, but just to flag, because in my experience people often conflate these things: Even if in some sense your all-things-considered preferences need to be complete, this doesn't mean your preferences w.r.t. your first-order axiology need to be complete. For example, take the donation case. You might be very sympathetic to a total utilitarian axiology, but when deciding whether to donate, your evaluation of the total utilitarian betterness-under-uncertainty of one option vs. another doesn't need to be complete. You might, say, just rule out options that are stochastically dominated w.r.t. total utility, and then decide among the remaining options based on non-consequentialist considerations. (More on this idea here.)

Why do you consider completeness self-evident? (Or continuity, although I'm more sympathetic to that one.)

Also, it's important not to conflate "given these axioms, your preferences can be represented as maximizing expected utility w.r.t. some utility function" with "given these axioms [and a precise probability distribution representing your beliefs], you ought to make decisions by maximizing expected value, where 'value' is given by the axiology you actually endorse." I'd recommend this paper on the topic (especially Sec. 4), and Sec. 2.2 here.

I mean, it seems to me like a striking "throw a ball in the air and have it land and balance perfectly on a needle" kind of coincidence to end at exactly — or indistinguishably close to — 50/50 (or at any other position of complete agnosticism, e.g. even if one rejects precise credences).

I don't see how this critique applies to imprecise credences. Imprecise credences by definition don't say "exactly 50/50."

Up until the last paragraph, I very much found myself nodding along with this. It's a nice summary of the kinds of reasons I'm puzzled by the theory of change of most digital sentience advocacy.

But in your conclusion, I worry there's a bit of conflation between 1) pausing creation of artificial minds, full stop, and 2) pausing creation of more advanced AI systems. My understanding is that Pause AI is only realistically aiming for (2) — is that right? I'm happy to grant for the sake of argument that it's feasible to get labs and governments to coordinate on not advancing the AI frontier. It seems much, much harder to get coordination on reducing the rate of production of artificial minds. For all we know, if weaker AIs suffer to a nontrivial degree, the pause could backfire because people would just use many more instances of these AIs to do the same tasks they would've otherwise done with a larger model. (An artificial sentience "small animal replacement problem"?)