Hello! My name is Vaden Masrani and I'm a grad student at UBC in machine learning. I'm a friend of the community and have been very impressed with all the excellent work done here, but I've become very worried about the new longtermist trend developing recently.
I've written a critical review of longtermism here in hopes that bringing an 'outsiders' perspective might help stimulate some new conversation in this space. I'm posting the piece in the forum hoping that William MacAskill and Hilary Greaves might see and respond to it. There's also a little reddit discussion forming as well that might be of interest to some.
Cheers!
On the expected value argument, are you referring to this?
Based on the link to the wiki page for random variables, I think Vaden didn't mean that the probabilities themselves follow some distributions, but was rather just identifying probability distributions with the random variables they represent, i.e., given any probability distribution, there's a random variable distributed according to it.
However, I do think his point does lead us to want to entertain multiple probability distributions.
If you did have probabilities over your outcome probabilities or aggregate utilities, I'd think you could just take iterated expectations. If U is the aggregate utility, U∼p and p∼q, then you'd just take the expected value of p with respect to q first, and calculate:
EV∼Eq[p][V]]If the dependence is more complicated (you talk about correlations), you might use (something similar to) the law of total expectation.
And you'd use Gilboa and Schmeidler's maxmin expected value approach if you don't even have a joint probability distribution over all of the probabilities.
A more recent alternative to maxmin is the maximality rule, which is to rule out any choices whose expected utilities are weakly dominated by the expected utilities of another specific choice.
https://academic.oup.com/pq/article-abstract/71/1/141/5828678
https://globalprioritiesinstitute.org/andreas-mogensen-maximal-cluelessness/
https://forum.effectivealtruism.org/posts/WSytm4XG9DrxCYEwg/andreas-mogensen-s-maximal-cluelessness
Mogensen comes out against this rule in the end for being too permissive, though. However, I'm not convinced that's true, since that depends on your particular probabilities. I think you can get further with hedging.