Fwiw, I think DiGiovanni would happily press a button that, with certainty, removes all unwanted suffering from the universe forever and keeps all else equal. (This could be a first station on another train.)

(P.S. Thanks!)

One could also assume that no one ever will, but that they would if they were in an implausibly simple decision situation. Unconditional cluelessness is more radical than "no one will ever have a justified c-pref in the real world" (which is itself more radical than expansive cluelessness).

Interesting post and well-written!

donating $4.99 to MAWF and donating $5.01 to MAWF are also available actions, so your reflection would have to lead you to assign higher EV to donating $5 than to either of these. It’s hard to see how reflection could do that. Coming to believe that donating $5 is the highest-EV action available to you would seem to require forming beliefs about the likely consequences of donating $5 specifically.

Don't we have the exact same problem with any amount given to any org, such that all possible donations are Nowhere-Optimal according to your argument? What makes MAWF special?

Interesting. What would we need to change in your chess analogy for your argument not to hold anymore? E.g., if we assume we don't know where the opponent's pieces are or something? 

And is this changed analogy really less relevant to our real-world predicament than your original analogy?

"Where does this leave cluelessness about the far future?" Narrower than it might first look: I'm not claiming trimmed maximality resolves severe cluelessness or restores confident long-termist verdicts. Many far-future comparisons will likely still come out indeterminate even under a trimmed rule, since the disagreement in those cases runs through the core of the representor, not just its edges.

Any example of an action that your rule recommends doing? (i.e., one where indeterminacy is not so great that even your trimmed-maximality rule isn't action-guiding?)

Thanks Sarah, very helpful and pleasant to have your reaction :)

I disagree with the decision being made with respect to an epistemically idealized self, rather than with the information and reasoning ability available to me.

DiGiovanni advocates for with respect to your current expectation of what your epistemically idealized self would believe. This is nothing more than the reflection principle. If you don't think your idealized self believes X, why would you believe X? That'd be a violation of this very consensual principle.

(Idk how central this actually is to your overall case, but I wanted to react to that.)

Is your valuation of these virtues of cluelessness conditional on the research/discussions virtuous people do/have, not leading to bad outcomes? I.e., if you believed that those who act in accordance with these virtues, and hence did research or similar, systematically ended up with misguided research results telling them to do something that is actually overall bad, would you stop valuing these virtues?

I think your endorsement of the "canceling out" response to cluelessness is THE crux, and that all your objections might, in fact, ultimately be downstream of this.

And you motivate your endorsement of "cancel out" with a thought experiment where you have reasons to believe pile A is lighter than pile B + asymmetric reasons to believe the opposite + some "tie-breaking" pebble taken out of pile A.

My reasoning: If Alice draws an object at random from the set of objects (even if the distribution is non-uniform), then she’s more likely to draw from the left pile when the left pile is heavier. And if the left pile is heavier, then it’s probably not heavier by exactly one pebble, and therefore it’s still heavier after a pebble is removed.

I don't think that Alice drawing an object at random is a good analogy to prove "cancel out" works. The thing here is that your new evidence has overthrown your previous conflicting evidence. It is so much more relevant, so you didn't need to assume previous evidence canceled out. So, yeah, ofc, if you gave DiGiovanni a slam-dunk argument for why pausing AI is good, that overthrows all the conflicting evidence he was considering, he would agree with you and reject cluelessness.

But to clearly prove that the "cancel out" move is legit, you need the potential "tie-breaking" consideration to be something trivial that does not overthrow previous evidence, e.g., the wind weakly blows from pile B to pile A, making pile A slightly heavier with dust, all else equal.

Now, would you say the wind consideration is a tie-breaker such that you should now bet B is lighter, despite the prior conflicting (and arguably more significant) evidence you had?^[1] Then you'd actually be clearly assuming the prior conflicting evidence canceled out.

1. ^{^}

   Cancel out reasoning: 

   • the asymmetric reasons to believe A or B is lighter leave me with a precise 50% credence B is lighter. (This is what you're saying when you write "it’s reasonable to believe that the piles are balanced in expectation.")
   • new consideration in favor of B being lighter (the wind).
   • I update from 50% to slightly above 50% that B is lighter.

I worry that this changes the target of DiGiovanni’s argument. As I understand him, the challenge is aimed at the kind of impartial altruistic action-guidance many EAs care about: guidance concerning the overall effects of our actions, including distant and indirect effects.

So even if there are alternative uses of “impartial,” I’m not yet seeing why they are relevant to that target. 

The overall effects of my actions are exactly what I care about. If you want me to ignore some of those effects, I need a substantive argument for why that restriction is morally acceptable, not just the observation that there are possible alternative notions of impartiality.