Statistics PhD student based in London.

Personal website: athowes.github.io

2

Thanks for sharing Rob! Here's a summary of my comments from our conversation:

I think the terminology good vs "optimal" is a little confusing in that the probability of obtaining a future which is mathematically optimal seems to me to be zero. I'd suggest "great" instead.

Some futures, which I’ll call “optimal”, are several orders of magnitude better than other seemingly good futures.

(Using great rather than "optimal") I'd imagine that some great futures are also several orders of magnitude better than other seemingly great futures. I think here we'd really like to say something about the rates of decay.

Let be the value of the future, which we suppose has some distribution . It's my belief that would be essentially continuous variable but in this post you choose to distinguish between two types of future ("good" and "optimal") rather than consider a continuous distribution .

This choice could be justified if you believe a priori that is bimodal, or perhaps multi-modal. If this reflects your view, I think it would be good to make your reasoning on this more explicit.

One way to model the value of the future is

where refers to the physical, spatial, temporal resources that humanity has access to, are our ethical beliefs about how best to use those resources and the final term reflects our ability to use resources in prompting our values. In A2 your basic model of the future is suggestive of a multi-modal distribution over future . It does seem reasonable to me that this would be the case. I'm quite uncertain about the distributions on the other two terms which appear less physically constrained.

Events which are possible may still have zero probability, see "Almost never" on this Wikipedia page. That being said I think I still might object even if it was ϵ-optimal (within some small number ϵ>0 of achieving the mathematically optimal future) unless this could be meaningfully justified somehow.