Sometimes, as a way for me to get more strategic clarity about what intermediate goals I want my research to accomplish, I try to evaluate whether things that are locally good/bad are good for the long-term future. For example, technological growth, economic growth, forecasting, IIDM, democracy, poverty reduction, non-existential catastrophic risks, and so forth.
My standard argument/frame of thinking goes like this:
“Well, you start with a prior of ~50-50 that any given macroscopic change is good for the long-term future, and then you update on the evidence that-”
And if this is done in conversation, my interlocutor often interrupts me with
“50-50 is a crazy prior because-”
And often it’s some argument that locally good things should be expected to be globally good. Sometimes people reference flow-through effects.There’s different flavors of this, but the most elegant version I’ve heard is “duh, good things are good.”
And like, I sort of buy this somewhat. I think it’s intuitive that good things are good, and I’ve argued before that we should start with an intuition that first-order effects (for a specific target variable) are higher than second-order effects. While that argument is strongest about local variables, perhaps we should expect that generally there’s a correlation between a thing’s goodness on one metric to its goodness on other metrics (even if the tails come apart and things that are amazing for one metric aren't the best for other metrics).
But when it comes to the long-term future, how much should we buy that things that are considered good by near-term proxies that don’t consider the long-term future are good for long-term stuff?
Put another way, what’s our prior that “an arbitrarily chosen intervention that we believe to be highly likely to be net positive for the experience of sentient beings in the next 0-5 years increases the likelihood of P(utopia)?"
--
Related post (though I don't exactly agree with the formalisms)
I came here to say this--in particular that I think my prior probability for "good things are good for the long-term future" might be very different than my prior for "good things are good for the long-term future in expectation", so it matters a lot which is being asked.
I think the former is probably much closer to 50% than the latter. These aren't my actual estimates, but for illustrative purposes I think the numbers might be something like 55% and 90%.
I agree with Eli that my actual estimates would also depend on the other questions Eli raises.
Another factor that might affect my prior a lot is what the reference class of "good things" looks like. In particular, are we weighting good things based on how often these good things are done / how much money is spent on them, or weighting them once per unique thing as if someone were generating a list of good things? E.g. Does "donation to a GiveWell top charity" count a lot, or once? (Linch's wording at the end of the post makes it seem like he means the latter.)
Perhaps it would be helpful to Linch's question to generate a list of 10-20 "good things" and then actually think about each one carefully and estimate the probability that it is good for the future, and good for the future in expectation, and use these 10-20 data points to estimate what one's prior should be. (Any thoughts on whether this would be a worthwhile research activity, Linch or others reading this?)