This is the third in a series of posts exploring consequentialist cluelessness and its implications for effective altruism:
- The first post describes cluelessness & its relevance to EA; arguing that for many popular EA interventions we don’t have a clue about the intervention’s overall net impact.
- The second post considers a potential reply to concerns about cluelessness.
- This post examines how tractable cluelessness is – to what extent we can grow more clueful about an intervention through intentional effort?
- The fourth post discusses what being clueless implies about doing good.
Consider reading the first and second posts first.
Let's consider the tractability of cluelessness in two parts:
- How clueful do we need to be before deciding on a course of action? (i.e. how much effort should we spend contemplating & exploring before committing resources to an intervention?)
- How clueful can we become by contemplation & exploration?
How clueful do we need to be before deciding on a course of action?
In his talk Crucial Considerations and Wise Philanthropy, Nick Bostrom defines a crucial consideration as “a consideration such that if it were taken into account it would overturn the conclusions we would otherwise reach about how we should direct our efforts, or an idea or argument that might possibly reveal the need not just for some minor course adjustment in our practical endeavors but a major change of direction or priority.”
A plausible reply to “how clueful do we need to be before deciding on a course of action?” might be: “as clueful as is needed to uncover all the crucial considerations relevant to the decision.”
Deciding to act before uncovering all the crucial considerations relevant to the decision is potentially disastrous, as even one unknown crucial consideration could bear on the consequences of the decision in a way that would entirely revise the moral calculus.
In contrast, deciding to act before uncovering all non-crucial (“normal”) considerations is by definition not disastrous, as unknown normal considerations might imply a minor course adjustment but not a radically different direction.
How clueful can we become by contemplation & exploration?
Under this framing, our second tractability question can be rephrased as “by contemplation and exploration, can we uncover all the crucial considerations relevant to a decision?”
For cases where the answer is “yes”, we can become clueful enough to make a good decision – we can uncover and consider everything that would necessitate a radical change of direction.
Conversely, in cases where the answer is “no”, we can’t become clueful enough to make a good decision – despite our efforts there will remain unknown considerations that, if known, would radically change our decision-making.
There is a difference here between long-run consequences and indirect consequences (see definitions in the first post). By careful investigation, we can uncover more & more of the indirect, temporally near consequences of an intervention. It’s plausible that for many interventions, we could uncover all the indirect consequences that relate to the intervention’s crucial considerations.
But we probably can’t uncover most of the long-run consequences of an intervention by investigation. We can improve our forecasting ability, but because of the complexity of reality, the fidelity of real-world forecasts declines as they extend into the future. It seems unlikely that our forecasting will be able to generate believable predictions of impacts more than 30 years out anytime soon.
Because many of the consequences of an intervention unfold on a long time horizon (one that’s much longer than our forecasting horizon), it’s implausible to uncover all the long-run consequences that relate to the intervention’s crucial considerations.
Ethical precautionary principle
Then, for any decision whose consequences are distributed over a long time horizon (i.e. most decisions), it’s difficult to be sure that we are operating in the “yes we can become clueful enough” category. More precisely, we can only become sufficiently clueful for decisions where there are no unknown crucial considerations that lie past our forecasting horizon.
Due to the vast size of the future, even a small probability of an unknown, temporally distant crucial consideration should give us pause.
I think this implies operating under an ethical precautionary principle: acting as if there were always an unknown crucial consideration that would strongly affect our decision-making, if only we knew it (i.e. always acting as if we are in the “no we can’t become clueful enough” category).
Does always following this precautionary principle imply analysis paralysis, such that we never take any action at all? I don’t think so. We find ourselves in the middle of a process that’s underway, and devoting all of our resources to analysis & contemplation is itself a decision (“If you choose not to decide, you still have made a choice”).
Instead of paralyzing us, I think the ethical precautionary principle implies that we should focus our efforts in some areas and avoid others. I’ll explore this further in the next post.
Thank you for this series — I this is is an enormously important consideration when trying to do good, and I wish it were talked about more.
I am rereading this, and find myself nodding along vigorously to this paragraph:
But not the following one:
Perhaps we indeed should move towards "analysis paralysis", and reject actions that we do not have a very high level of certainty in the long-term effects of. Given the maxim that we should always act as if we are in the "no we can't become clueful enough" category, this approach would reject actions that we anticipate to have large long-term effects (e.g. radically changing government policy, founding a company that becomes very large). But it's not clear to me that it would reject all actions. Intuitively, P(cooking myself this fried egg will have large long-term effects) is low.
We can ask ourselves whether we are always in the position of the physician treating baby Hitler: every day when we go into work, we face many seemingly inconsequential decisions that are actually very consequential. i.e. P(cooking myself this fried egg will have large long-term effects) is actually high. But this doesn't seem self-evident.
In other words, it might be tractable to minimize the number of very consequential decisions that the world makes, and this might be a way out of extreme consequentialist cluelessness. For example, imagine a world made up of many populated islands, where overseas travel is impossible and so the islands are causally separated. In such a world, the possible effects of any one action end at the island it started at, so therefore the consequences of any one action are capped in a way they are not in this world.
It seems to me that this approach would imply an EA that looks very different than the current one (and recommendations that look different than the ones you make in the next post). But it may also be a sub-consideration of the general considerations you lay out in your next post. What do you think?
Bostrom defines a "crucial consideration" as one that would overturn a conclusion or reveal the need for a major change of direction. By this definition, something may or may not be a "crucial consideration" depending on our current set of conclusions and our current direction. The definition sneaks in a connotation that important new insights will tend to reveal the need for a major change of direction. But it's also possible that important new insights will reaffirm our current direction. See conservation of expected evidence.
Regarding the precautionary principle, consider a reversibility test: Suppose there is some parameter of the world p which is gradually increasing, and you have the opportunity to interfere and stop this increase for no cost. By the precautionary principle, you should not interfere. Now suppose p is currently static, and you have the opportunity to interfere and trigger a gradual increase for no cost. Again, by the precautionary principle, you should not interfere.
For someone like me, who does not believe in the act/omission distinction and believes in fighting status quo bias, this seems a little silly. I think the best arguments for a policy of non-interference in both scenarios are:
I'm sympathetic to the idea that value of information is high, and I think cluelessness is tractable. I support EA groups like the Future of Humanity Institute which are trying to work out the best course of action. But at a certain point, the low-hanging information fruit will get picked, and then it's likely time to act. If we aren't going to take action under any circumstances, gathering information is a waste of time.