From_Cluelessness_to_Clarity
From Cluelessness to Clarity: With a Different Question
TL;DR
Anthony DiGiovanni argues that there is no justified reason to prefer pausing AI over continuing it, or the other way around, because neither option wins under every possible way the future could go. That's true under his rule. But his rule is a choice, not a fact about the world—and there's a different rule we can use instead. The alternate rule compares the worst case of each option, instead of demanding a win everywhere, which gives real answers in most cases. It runs into the same tie in the AI-pause example, but it takes you further into understanding why the tie holds.
A constructive response to Anthony DiGiovanni's "The Challenge of Unawareness for Impartial Altruist Action Guidance".
Entry type
Constructive response (Option 3). Anthony DiGiovanni's own rule often can't establish a better option. This response argues for a different rule. Using the same foggy information, this rule sometimes leads you to a real answer, which the earlier rule could not.
This paper also proposes a concrete answer to Anthony DiGiovanni's concern about what standard a person trying to do good should actually follow to make a decision.
“When we are not sure about any option and worried about human good in general, then we should not look for what good can happen—we must look for what is less bad that we can’t avoid. Because under uncertainty, the degree of bad impacts more.”
For context
Anthony DiGiovanni argues that when we try to make an impartial, long-term decision for everyone's good, our understanding of the consequences is often too uncertain to justify it. He calls this the problem of "cluelessness".
1. Anthony DiGiovanni's argument, briefly
Anthony DiGiovanni asks, "What is the better way to approach the future?" Is it by pausing AI development or by continuing it further?
Pausing AI gives safety research more time to catch up with its power. It provides time for governance, law, the public, and the rest of the world to prepare themselves for something new.
But pausing can't guarantee that reckless actors will actually follow it. The work itself doesn't stop—it just piles up unreleased. When the pause ends, all of that built-up work gets released at once, with less time to handle it carefully than if things had moved forward steadily.
So, both actions make sense. We can't say one is clearly right. And DiGiovanni argues that we don't have a justified reason to prefer one option over the other.
These arguments are justified, and we don’t have a strong reason to follow either option. DiGiovanni just picked a specific way to handle it. But I think this doesn’t have to be the only option.
2. The rule that produces ties — and an alternative
Anthony DiGiovanni says one option is better than the other if it comes out ahead in every single scenario [1]. If pausing AI looks better in some scenarios and non-pausing looks better in the other scenarios, then neither one is better for all scenarios. So, they tie. There is no justified reason to choose either option.
His rule seems to be a little strict. When a single scenario in favour of the weaker option is enough to produce a tie, we can almost anticipate a “no answer” every time the rule is used. This strictness is not introduced by the fog itself. This is a choice DiGiovanni made about what counts as a good enough reason to prefer one option over another.
There is an alternative—a different question. If things go as bad as they can, we need to understand how much worse the worst case gets under each choice. Which worst-case is less bad for the people in general?
And to get clarity, we need to ask, "Which worst case is less bad?" The response to this comparative question always has an answer in a foggy scenario, whereas the earlier question "Who wins everywhere?" never gives a binary answer in the same situation.
3. We can never list every possible future
When we try to answer the worst that can happen while examining each choice, we can only answer it from our own perception of what the worst could be. We will never be able to find the real worst. There is always a possibility of something worse that nobody has thought of that could still happen. So, this worst case is the worst case we have imagined.
But the same problem doesn’t stop Anthony DiGiovanni. In his post, he says, “Of course, we don’t always need to be aware of every possible outcome to make justified decisions" [2].
He accepts that for local goals in a familiar domain an incomplete map is absolutely fine. But for long-term decisions like AI safety, where the future is vast, he argues we cannot take any justified decision without complete awareness.
But becoming fully aware of everything required to make a decision is practically impossible. I believe awareness is not about how much we know; it’s about the depth of our understanding about what we know.
So, our rule needs that kind of awareness, with depth. Before we decide, we find the worst case, observe its impact, and act on it. And it’s not all; we keep working on it till further actions won’t improve its impact. This is where awareness is not just an observation; it becomes an action.
The contrast with DiGiovanni: His approach, without complete awareness, means we are not sure about the decision; we stop, and things stall. With our approach, with limited awareness, we observe, we act, try to improve, and then decide and keep marching ahead.
4. The depth of worst case
Worst case is not what intuitively comes to mind when we start thinking about the worst that can happen. It’s just the beginning. The real worst-case lives in the depth; we need to extract it. This extraction is done by analysing different angles and exploring different aspects by probing it with negative questions. Also, getting into the depth is not a linear process. It needs to be done iteratively over time to reach a real depth.
Example: Consider deciding whether to leave your current job with the assumption of getting a new job within 6 months.
Initial worst-case: "What if I don't get the job within 6 months?” - Your instinct immediately responds with “I have enough money to survive." It’s too early to conclude—we need to ask negative questions to explore it further.
All these questions explore the hidden assumptions and dimensions of the worst case by asking those negative questions.
And again, when we don’t get a job even after 1 year, it brings new challenges like the current state of paying bills, education fees, and medical expenses, which cannot be fully weighed at the time of the original decision.
The way we get to this depth is by finding the assumptions that are hidden inside each worst case and then asking, “What if each one fails?”
5. Severity matters more than frequency
While comparing the worst-case scenarios, our mind naturally falls into a trap. When we see an option with one worst-case scenario, it naturally feels better than the option with multiple worst-case scenarios.
The moment we start counting the number of worst-case scenarios for each option, unknowingly, we weigh each scenario on the same scale. In some cases, the same scenario might be split into multiple scenarios. This is totally driven by how the user frames the scenario.
A decision taken based on the number of worst-case scenarios doesn’t make sense, because these scenarios influence lives differently. So, the worst-case scenarios need to be evaluated based on severity.
Let’s see one example: Illness
Option A (take medicine): worst case — longer hospital stay. Painful, costly, etc.
Option B (don't take medicine): worst case — death
If we consider the counting scenarios, option B wins, as it has only one worst-case scenario—which is clearly not the right choice.
But when we consider the severity of the worst case, how bad it actually gets, we see that option A stands out. With option B—no medicine—we reach the worst-case death, which is irreversible, but when we take medicine, option A, in the worst case, we stay longer in the hospital, expenses will be higher, and the process is painful, but ultimately, we recover, which is the natural choice in this case.
So, it doesn’t matter how many bad scenarios each option has; what matters is how bad the worst scenario can actually get.
Sometimes a worst-case scenario is not static—it gets worse when left unattended. A leak in the bucket makes the bucket empty if not fixed. An issue that is fixable today might not be fixable tomorrow. So, the answer to "how bad is the worst case" should not be limited to the current time frame—it has to be answered by anticipating the future.
6. The worst case isn't fixed—we can shrink it
Until now, the way we have discussed the worst case feels like it is fixed and static. In the earlier example where one of the worst-case scenarios is "painful", it looks like there's nothing we can do about it.
But actually, in this case we act on this scenario, like giving some additional painkillers, diverting the mind from pain by being present, or providing gadgets to distract from the pain—all of these help in making it less severe.
So, how bad a worst-case scenario can be sometimes depends on what can be done about it.
6.1. The concept of floor: how far the worst case can shrink
In the previous section we saw that the worst-case scenario is not fixed and can be improved by acting on it. But we will not stop just by acting once. We will keep working on it till it reaches a certain point, beyond which we don’t see any further improvement. The point where the worst case stops improving—we call this the 'floor'.
The floor is not a static point. It is a dynamic point, explored by continuously improving the worst case until we reach the least harm possible.
When we have many actions to be applied, we keep applying the action till we reach a point where we don’t see further improvement. There is no need to apply any further action once we reach the floor.
When our actions are exhausted, but we still see the improvement, yet we don’t really have any further actions to be applied, at this point, wherever we have reached, we call that a 'floor'.
So, "floor" doesn't mean we have absolutely reached the bottom; it means we have stopped our efforts to improve it further. The floor we reach is not an absolute number but our observation that the improvement has stalled—it is our logical decision to stop.
6.2. Significance of floor
While improving the worst-case scenario, the floor gives us an idea of how much effort has gone into the improvement. Combining all the efforts made to improve all the worst cases brings understanding and builds confidence for choosing one option over the other.
So, while getting to the floor, we need to push reasonably hard with the intention of getting the most improvement for the worst case. When we stop, we will be able to explain the absolute logical reason for the halt.
Sometimes reaching the floor is not possible because we can’t come up with all possible actions upfront. But that doesn’t stop us from applying the actions. We try to improve the worst case with the actions available to us.
Again, when both options reach their respective floor, that doesn’t mean there’s a tie. Understanding the actions taken across both options gives us the confidence to choose one option over the other.
7. Testing the method on Anthony DiGiovanni's own example
When we don’t pause: In the worst case, a misaligned system will be built, deployed, and used before anyone identifies the issues.
When we pause, in the worst case, the reckless actors will still be building and using the misaligned systems. However, the capabilities that get built by the responsible actors during the pause will be released all at once when the pause is over. This gives less time to handle it carefully and, in a way, makes the situation misaligned as well.
Comparing the severity of these two worst cases, we find they reach the same place — a tie. We are not claiming that our rule will break the tie, but with our concept of the floor, the inherent tie gets some logical direction to move ahead.
7.1. Additional costs: towards more worst-case scenarios
With our rule, the worst case is not the only cost. What we miss along the way also makes some difference. This miss is a hidden cost we pay while navigating towards the decision.
The additional costs that we pay for a pause are "lost momentum", “lost revenue", “lost continuity", etc. For non-pausing, the apparent costs are "missing the time to recover", "missing the chance to avoid the harm", etc.
The additional costs we pay while pausing add to our worries by contributing to the worst cases, whereas the additional cost for non-pausing turns out to be the same catastrophic harm, just described from a different angle.
7.2. Pushing the choices to their floors
When we encounter a tie, the concept of a floor can give direction. To get a strong direction, we need to see how much we can push each worst-case down with practical, real-life actions.
If we choose non-pause, in the worst case, development continues, and in this case the floor can be pushed down by doing the safety work, like testing, monitoring, staged release, etc., along with the development.
If we choose to pause, we can build the floor by doing things, which helps reduce the harm. This can be done with the tools we develop, the agreement we make, the rules we define for safety and control, etc., during the pause.
With the knowledge of the floor, the comparison turns a blind spot into real insight and helps guide the decision. But we can’t come up with those real-life actions that can push the floor to its limit on our own. We need insights from the relevant people in the industry.
We landed in a tie with “cluelessness” by asking a question earlier, but by asking a different question, we still landed in a tie, but this time with “some clarity" on where we stand with the help of the floor.
8. What this means in practice
In practice, this alternate rule remains focused on the outcomes, not by choosing what's good but by avoiding the maximum bad.
The way we apply this in practice is by:
This method doesn’t give an answer all the time, but whenever it hits a tie, it will always empower you with the knowledge to understand the tie. This is because of the path the method takes to reach this point of cluelessness. And this knowledge helps you find a justified reason to drop your shrug and choose the safer option.
Waiting at a dead end has its own consequences — it will not take us anywhere. We need to take some action. It doesn’t matter, however small it is. We need to pick one of the directions and keep working on it to improve. While picking the direction, ensure it gives enough room to switch direction when required. When we get a clearer signal along the way, we adjust the direction, and we will ultimately get closer to a solution.
When multiple stakeholders are involved in decision-making, selecting an option for everyone’s good is tricky. We need to explore the challenges when the perception of the worst-case changes across different stakeholders.
References
[1] DiGiovanni, Anthony. "The Challenge of Unawareness for Impartial Altruist Action Guidance: Introduction." EA Forum, June 2, 2025. https://forum.effectivealtruism.org/s/rHqdsrieinyhM5KDv/p/a3hnfA9EnYm9bssTZ
[2] Ibid.
Disclosure
Ideas in this paper were developed through dialogue with an AI assistant. All arguments, examples, and written text are the author's own. AI was used as a thinking partner and editor, not as a writer.
