I’d like to elicit direct, productive critiques of the argument for cluelessness from my sequence on “unawareness”, which I’ll call the unawareness argument.

To that end, this post will:

• break down the unawareness argument at a high level;
• explain why the EA community should care about this argument; and
• summarize the angles for critiquing the argument that I expect to be most productive, and how the sequence responds to existing critiques.

Argument breakdown

Here’s a new framing of the unawareness argument (compared to how I present it in the sequence). I expect this framing to help readers disentangle different types of disagreements they might have, corresponding to three different premises of the argument.

Roughly: What would justify preferring action A over B on impartial altruistic grounds? We’d need to “expect” that according to our epistemically idealized self, A has better expected total consequences across the cosmos (normative premise). But if our understanding of these actions’ consequences is too coarse, then we can’t say how our idealized self would compare their expected values (conceptual premise). And our understanding of any given action’s cosmos-wide consequences is in fact that coarse (empirical premise). So there’s no impartial altruistic justification for preferring any action over another.

More precisely:

• Let’s say that we c-prefer^[1] A over B if the reason we prefer A is an impartial altruistic comparison of the actions’ possible consequences.
• P1. Normative premise: To justify c-preferring A over B, it’s not enough to say (e.g.) that A seems heuristically good. Rather, we need to argue that A has higher “expected value” broadly speaking, meaning: In some sense we “expect” that, if we were idealized agents who could aggregate all of A’s and B’s possible consequences into literal EVs, then we’d say A has higher EV. (We ourselves don’t need literal EVs to justify c-preferences, hence the scare quotes.^[2]) Otherwise, it’s unacceptably arbitrary to c-prefer A.
• P2. Conceptual premise: If our understanding of A’s and B’s possible consequences is sufficiently coarse-grained, then we don’t have an argument for “expecting” our idealized self’s EV for A to be higher, lower, or equal to B’s.^[3] So A’s and B’s “EVs” are incomparable. In particular:
  • a. Against precise EVs: We shouldn’t represent actions’ degrees of c-preferability with literal precise expected values.
  • b. Against “best guesses”: Even if we don’t use literal precise EVs, we shouldn’t always force ourselves to compare A’s and B’s “EVs”.
• P3. Empirical^[4] premise: Due to unawareness (at least), our understanding of any pair of actions’ possible consequences is indeed very coarse-grained — enough that the conclusion of (P2) follows (i.e., these actions’ “EVs” are incomparable). In particular, the actions’ “EVs” are too severely imprecise to compare them, regardless of whether we (a) formally model these “EVs” or (b) appeal to informal/heuristic arguments.
• Conclusion: We aren’t justified in c-preferring any action over any other.

(See Appendix for how each section of the sequence maps onto this form of the argument.)

Why cluelessness matters

Here’s a natural reaction to this argument: “If we’re clueless, nothing we do matters anyway. So there’s no point looking into the argument, and we should just act as if we aren’t clueless.”

But I don’t think it’s that simple. I’ll explain why, then say what kinds of “looking into the argument” I expect to have the highest value of information.

First, the unawareness argument doesn’t imply that “nothing we do matters” all things considered. It only implies that impartial altruism, or any very far-reaching value system, isn’t action-guiding. Other values and moral norms still matter to us, for example, rules like avoiding dishonesty or virtues like compassion. These can be action-guiding even if we’re clueless about total consequences.

Second, if you think the argument goes wrong somewhere, it makes a difference where it goes wrong:^[5]

• Do you reject the normative premise, because you think it counts as an impartial altruistic justification if we say “This action has good ‘expected’ consequences after bracketing the consequences we’re clueless about”? Then arguably you should prioritize neartermist causes.
• Do you accept the normative premise but reject the conceptual one, implying that we should form “best guesses” about the balance of all cosmos-wide consequences? Then you should look for interventions that are best after accounting for as many “galaxy-brained” considerations as possible, rather than simply ignore those considerations. (It’s been argued that mainstream x-risk reduction meets that bar — e.g., Shulman; Adelstein; Carlsmith — but I think this should be spelled out a lot more carefully.)
• Do you agree with the normative and conceptual premises, but think some cautious or “meta” interventions are justified without arbitrary calls about the considerations we’re unaware of? Like, say, saving resources until we’re in a clearer epistemic situation? Then you should do those interventions, rather than various other popular interventions whose justification does rely on arbitrary calls.

Third, relatedly, unawareness probably has some implications for impartial altruists, even if we don’t think it makes us clueless. There’s older work on crucial considerations, but I don’t think such work has rigorously fleshed out the implications much, relative to the scale of the problem. As an example of such an implication: Suppose we want to make forecasts about post-ASI civilization, but we’re worried that forecasting methods that worked well in better understood systems won’t generalize to this case. We could study various methods in domains with (e.g.) different frequencies of past crucial consideration discovery, and see which methods are relatively robust as the frequency of crucial considerations increases.^[6]

Finally, if nothing else, it seems epistemically virtuous to be clear about the reasons for our decisions. Sure, perhaps there’s no behavioral difference between “I’m working on AI risk because I’ve really weighed up all the possible consequences, and it doesn’t seem arbitrary to say this work is impartially good ‘in expectation’”, and “I have no clue if my idealized self would favor working on AI risk, but I’m doing it because no one has offered something better”. But I think if we’re honest with ourselves that our reasoning is the latter, we’ll have more open minds if and when “something better” comes along.

Critiques: What I expect to be productive, and what’s been said so far

So, if the unawareness argument is worth engaging with, what should we focus on when scrutinizing it? Going forward, I expect newer or sharper critiques of the normative premise, and part (b) of the conceptual premise, to be most productive. This is because:

• I feel relatively confused about these premises, and they seem underexplored, at least in precise terms. I’m not aware of many writings explicitly aiming to answer: “What is the standard that non-idealized impartial altruists (should) use to judge which actions are rational? If it’s ‘approximating EV’, or ‘going with our best guess, even if not with literal precise EVs’, what exactly do these things mean, and what justifies them?” (See also Violet Hour’s discussion of this open problem.)
  • I think a relatively promising framework to start with here is reasons-based choice.
• By contrast:
  • Part (a) of the conceptual premise is more crisp and therefore less confusing. There has already been a lot of academic work on the arguments for and against precise Bayesianism (for example).
  • Disagreements about the empirical premise partly hinge on different intuitions about, e.g., how we should extrapolate from our history of discovering certain backfire risks. These intuitions can be hard to make legible. (That said, we can still make progress by considering arguments like those in posts #3 and #4 of the unawareness sequence.)

As far as I’m aware, almost all existing critiques of the unawareness argument are addressed by the sequence, or by other references in this resource guide — see the following table. (The exception, included in the table as well, is the critique that incomparability violates decision-theoretic deference principles.)

Premise of the unawareness argument being responded to	Summary of critique (source)	Summary of response (corresponding section of the sequence, and other references)
(P1) Normative	No two actions are ever incomparable, because we always have to choose something. (Soares;[7] Bergman[8])	Even if we’re forced to choose something, this doesn’t tell us whether we have impartial altruistic reasons to choose A or B. (2.4, response to Q6 and links therein.)
(P1) Normative	We don’t need to explicitly compare actions’ “EV”. Instead, we can compare actions using heuristics. (Thorstad and Mogensen 2020)	By itself, “this heuristic favors action A” doesn’t tell us why A is c-preferable. We need to say why we believe this heuristic tracks A’s expected consequences from our idealized self’s perspective.[9] (1.1.2; see also “Winning isn’t enough”, section “Heuristics”, and “In Defence of Modeling”, section “Everyone is Modeling, Even if Implicitly”.)
(P1) Normative	Precise credences/EVs lead to better decisions in practice. (Lewis;[10] Shulman[11])	When this critique is spelled out, it seems to beg the question: It appeals to a standard of “better decisions” that’s either equivalent to precise EV maximization, or assumes a precise probability distribution.[12] (1.1.2; see also “Should you go with your best guess?”, section “Background on degrees of belief and what makes them rational”, and “Winning isn’t enough”, section “Heuristics”.)
(P1) Normative	Some actions are obviously c-preferable to others (not just preferable all things considered). So we should reject any philosophical argument to the contrary (“one person’s modus ponens is another’s modus tollens”). (Chappell; Mogensen)	C-preferability is (arguably) not something we can directly perceive. Rather, it is constituted by weighing up possible consequences. So the justification for our beliefs about c-preferability depends on the justification for our beliefs about the consequences.[13] And because the set of consequences we need to weigh up is extremely complex, we can’t trust our intuitions about the bottom-line verdict “the weight of consequences favors A”. (2.3; see also “How to not do decision theory backwards”.)
(P1) Normative	If you consider actions incomparable (and hence your preferences violate the completeness axiom), you can get money-pumped. (Gustafsson 2022)	The money pump argument against incomparability assumes the “decision-tree separability” principle. That principle is far less plausible than premise P1, which — for impartial altruistic judgments, given the other premises — implies incomparability. (2.4, response to Q5 and links therein; see also Bradley and Steele (2014), and Thoma’s (2024) response to money pump arguments generally.)
(P1) Normative	Incomparability violates very plausible deference/reflection principles. (Hare 2010; Tarsney et al. 2025)	These particular deference principles don’t tell us about A’s or B’s “expected” consequences themselves, in our actual epistemic state. They don’t answer P1’s question: “Which option has higher ‘EV’, and why do we think this?” So these principles don’t seem to justify c-preferring either option. This point applies to the money pump argument as well. (Not directly addressed by the sequence; but see this comment, and the conclusion of “How to not do decision theory backwards”.)
(P2.a) Conceptual: Against precise EVs	Unawareness isn’t a fundamental challenge to precise EV any more than regular uncertainty is. With both unawareness and uncertainty, we lack information about the possible consequences — but we can still make tradeoffs between these possibilities. (Greaves & MacAskill 2025, Sec. 7.2)	The disanalogy is: Under uncertainty, we can precisely specify the possible outcomes we’re making tradeoffs between. But under unawareness, we can’t. So, since our values as impartial altruists are defined over very fine-grained possible outcomes, precise EVs aren’t well-defined. (1.1.1; see also Roussos (2021).)
(P2.a) Conceptual: Against precise EVs	The motivation for rejecting precise EV is really just discomfort with committing to a number. This is a psychological difficulty, not an argument against precision. (Greaves;[14] Soares[15])	The worry about assigning precise EVs (with respect to impartial values) isn’t that it’s difficult, but that it’s arbitrary: we have no reason to pick one precise EV over many others. (2.1; see also “You probably already like imprecise probabilities”.)
(P2.a) Conceptual: Against precise EVs	The critiques of precise EV are based on the fact that we aren’t idealized Bayesian agents. This is true, but it doesn’t make precise EV maximization the wrong normative standard of rationality.[16] (Greenblatt)	We need to define our normative standard in terms of our epistemic situation. In that situation, precise EVs with respect to impartial values aren’t well-defined. And again, any particular precise EV assignment would be arbitrary even if we could define it (e.g., by assigning a precise utility to some “catch-all” outcome). (1.1.1, 2.1; see also this comment.) (It’s also not clear that an idealized agent should assign actions precise EVs; see Appendix of “Should you go with your best guess?”.)
(P2.b) Conceptual: Against “best guesses”	Our intuitions about which actions are c-preferable are at least slightly better than chance. That is,[17] they’re positively correlated with the ground truth of “what we’d c-prefer if we could explicitly aggregate all the possible consequences”. That’s enough to always be able to say which action is c-preferable. (Lewis[18])	We don’t have direct evidence that our intuitions about c-preferability tend to track truth. (2.3.1.1.) So we need to weigh (i) the weak positive evidence from (e.g.) near-term forecasting research, against (ii) other considerations, namely: First, our intuitions about c-preferability might systematically track things other than the truth (e.g., sources of bias in the sample of hypotheses that occur to us). (3.2.1.) Second, we should also put some weight on explicit models, which need to account for an extremely complex set of consequences. (3.2.) The problem is that it’s ambiguous how to weigh up (i) and (ii).[19] (2.1, 2.4.)
(P3.a) Empirical (a)	Even if our impact is dominated by consequences we’re unaware of, we don’t know which direction they point. So, subjectively we should regard the negative and positive consequences we’re unaware of as canceling out in expectation. (MacAskill[20]; Soares[21])	It doesn’t follow from “we don’t know the net direction of the consequences we’re unaware of” that we should regard the positives and negatives as precisely symmetric. One reason symmetry is implausible: If we become aware of a new possible consequence, this should update our beliefs about the others we’re unaware of, breaking the symmetry. (4.1.1.)
(P2.b) Conceptual: Against “best guesses”, (P3.b) Empirical (b)	Even in relatively simple real-world decision problems, as bounded agents, we face the same qualitative epistemic challenges the unawareness argument appeals to: We can’t form precise models of the consequences, we’re unaware of some things, etc. Presumably, we’re not clueless in those problems. So it’s not clear why we’d be clueless about the impartial good. (Ngo)	Premise P2 claims that actions are incomparable when our understanding of a decision problem is coarse-grained enough. There’s a principled threshold for “enough”, namely, when the comparison seems sensitive to arbitrary choices about how to fill in the details of our coarse-grained model. (3.1.1.) Unlike much simpler problems, promoting the impartial good is past this threshold — we have a much weaker understanding of the relevant mechanisms and a worse history of sign-flipping considerations. (2.2, 2.3; see also this comment.)

Acknowledgments

Thanks to Toby Tremlett, Clare Harris, and Konrad Kozaczek for comments.

Appendix: Sequence summary annotated with the corresponding premises

Here’s a copy of the unawareness sequence summary from post #1, where each section is tagged with the premises of the argument supported by that section.

1. The challenge of unawareness for impartial altruist action guidance: Introduction:

1. Unawareness consists of two problems: The possible outcomes we can conceive of are too coarse to precisely evaluate, and there are some outcomes we don’t conceive of in the first place. (1.1)
   • (P3): makes precise a basic sense in which our information about total consequences is coarse-grained.
2. Thus, unlike standard uncertainty, under unawareness it’s unclear how to make tradeoffs between the possible outcomes we consider when making decisions. (1.1.1)
   • (P2.a): blocks the most straightforward motivation for precise EVs.
3. We can’t dissolve this problem by avoiding explicit models of the future, or by only asking what works empirically. (1.1.2)
   • (P1): argues that we need some all-things-considered model of the consequences, not just (e.g.) brute appeals to heuristics.
4. A vignette illustrates how unawareness might undermine even intuitively robust interventions, like trying to reduce AI x-risk. (1.2)
   • (P3): concretely illustrates how our information about some longtermist intervention seems very coarse-grained.

2. Why intuitive comparisons of large-scale impact are unjustified:

1. The generalization of “expected value” to the case of unawareness should be imprecise, i.e., not a single number, but an interval. This is because assigning precise values to outcomes we’re not precisely aware of would be arbitrary. This imprecision doesn’t represent uncertainty about some “true EV” we’d endorse with more thought. Rather, it reflects irreducible indeterminacy: there is no single value pinned down by our evidence and epistemic principles. (2.1)
   • (P2.a).
2. Suppose we have (A) a deep understanding of the mechanisms determining a strategy’s consequences on some scale, and (B) evidence of consistent success in similar contexts. Then, we can trust that our intuitions factor in unawareness precisely enough to justify comparing strategies, relative to that scale. (2.2)
   • (P2.b), (P3.b): argues that (P2.b), (P3.b) don’t “prove too much”.
3. Unlike in everyday decision situations, we have neither (A) nor (B) when making choices based on the impartial good.
   1. Our understanding of our effects on high-stakes outcomes seems too shallow for us to have precisely calibrated intuitions. This is due to the novel and empirically inaccessible dynamics of, e.g., the development of superintelligence, civilization after space colonization, and possible interactions with other universes. (2.3.1)
      • (P2.b), (P3.b): argues that our tacit understanding of total consequences is coarse-grained, and hence “best guess” comparisons are inappropriate.
   2. The mechanisms we’re unaware of might be qualitatively distinct from those we’re aware of. They’re not merely the minor variations we know superforecasters can handle. (2.3.1.1)
      • (P2.b), (P3.b): rebuts a counterargument to the previous point.
   3. Instead of consistent success, we have a history of consistently fragile models of how to promote the impartial good. Based on EAs’ track record of discovering sign-flipping considerations and new scales of impact, we’re likely unaware of more such discoveries. (2.3.2)
      • (P2.b), (P3.b): argues that we’re likely unaware of many considerations of which we have only very coarse-grained understanding, and hence “best guess” comparisons are inappropriate.
4. If we don’t know how to weigh up evidence about our overall impact that points in different directions, then an intuitive precise guess is not a tiebreaker. This intuition is just one more piece of evidence to weigh up. (2.4)
   • (P1); (P2.a), (P2.b): rebuts “you have to choose something, so nothing is incomparable”; rebuts arguments for precise EVs and for “best guess” comparisons.

3. Why impartial altruists should suspend judgment under unawareness:

1. To get the imprecise “EV” of a strategy under unawareness, we take the EV with respect to all plausible ways of precisely evaluating coarse outcomes. Given two strategies, if neither strategy is net-better than the other under all these ways of making precise evaluations, then we’re not justified in comparing these strategies. (3.1; 3.1.1)
   • (P2.a), (P2.b): defends imprecise “EV”, and rebuts an argument for aggregating the set of “EVs” to make comparisons.
2. Our evaluations of pairs of strategies should be so severely imprecise that they’re incomparable, absent arguments to the contrary. This is for two reasons:
   1. Given the possibilities we’re aware of, there are very few constraints on how to precisely model the possibilities we’re unaware of. This lack of constraints is worsened by systematic biases in how we discover new hypotheses. For example, we may be disproportionately likely to consider hypotheses that we happen to find interesting. (3.2.1)
      • (P3.a): argues that our coarse-grained understanding of the catch-all implies severe imprecision.
   2. Suppose we try breaking down the space of possible outcomes into manageable categories. Since we can only break things down so far, the categories we can model remain too coarse-grained to pin down whether a strategy’s expected upsides outweigh its downsides. (3.2.2)
      • (P3.a): argues that our coarse-grained understanding of hypotheses we’re aware of also implies severe imprecision.
3. We have unawareness at the level of both (P1) how good different world-states are (like “misaligned AIs take over”) relative to each other, and (ii) how effective concrete interventions are at steering toward vs. away from a given world-state. (3.3.1)
   • (Framing for the argument in the next point.)
4. When we model the impact of the AI safety intervention from the vignette in (1d), the structural problems from (3b) and (3c) undermine the case for that intervention. That is, given reasonable ranges of parameter estimates, the intervention is positive under some estimates and negative under others, and it’s arbitrary how we weigh up their plausibility. (3.3.2)
   • (P3.a): supports the arguments about severe imprecision above with a specific formal example.

4. Why existing approaches to cause prioritization are not robust to unawareness:

1. We can’t assume the considerations we’re unaware of “cancel out”, because when we discover a new consideration, this assumption no longer holds. (4.1.1)
   • (P3.a): rebuts counterargument.
2. We can’t trust that the hypotheses we’re aware of are a representative sample (see 3.b.i), so we can’t naïvely extrapolate from them. Although we don’t know the net direction of our biases, this doesn’t justify the very strong assumption that we’re precisely unbiased in expectation. (4.1.2)
   • (P3.a): rebuts counterargument.
3. Similarly, we can’t trust that a strategy’s past success under smaller-scale unawareness is representative of how well it would promote the impartial good. The mechanisms that made a strategy work historically could actively mislead us when predicting its success on a deeply unfamiliar scale. (4.1.3)
   • (P3.a): rebuts counterargument.
4. The argument that heuristics are robust assumes we can neglect complex effects (i.e., effects beyond the “first order”), either in expectation or absolutely. But under unawareness, we have no reason to think these effects cancel out, and should expect them to matter a lot collectively. (4.1.4)
   • (P3.a): rebuts counterargument.
5. Even if we focus on near-term lock-in, we can’t control our impact on these lock-in events precisely enough, nor can we tease apart their relative value when we only picture them coarsely. The “punt to the future” approach doesn’t help for similar reasons. (4.1.5; 4.1.6)
   • (P3.a): rebuts counterargument.
6. Suppose that when we choose between strategies, we only consider the effects we can weigh up under unawareness, because (we think) the other effects aren’t decision-relevant. Then, it seems arbitrary how we group together “effects we can weigh up”. (4.2)
   • (P1): rebuts an argument that A can be c-preferable to B even if A’s total consequences aren’t better.

---

1. Cf. “c-betterness” from Greaves (2016). ↩︎

2. In other words, we need to appeal to something like EV — our beliefs about the (perhaps imprecise) EV our idealized self would compute — to give an impartial altruistic justification for some choice. In his post on “Ideal Reflection”, Clifton discusses a similar idea: “‘The’ expected value is the expected value that would be assigned by an agent that has the same evidence as us, but is a vastly more powerful reasoner. Something like a perfect Bayesian who can reason over a ~maximally granular and exhaustive set of hypotheses, and has seen everything we’ve seen.” Clifton also notes, and I agree, that it’s not clear what exactly it means for us to have “expectations”-in-scare-quotes about our idealized self’s EVs; see his footnote 1. But as far as I can tell, this vague notion of “expectation” captures the kind of aggregation of possible consequences that impartial altruists aspire to, and that EAs typically appeal to. ↩︎

3. In the unawareness sequence, this claim largely maps onto the claim that we should represent actions’ “EV” imprecisely, to some degree. (And then, the remaining question is whether the degree of imprecision is so severe that all actions are incomparable.) But the argument doesn’t rely on the particular formal model of imprecise probabilities, or sets of expected values. ↩︎

4. “Empirical” in the sense that P3 is largely about contingent facts of our actual epistemic situation. But it isn’t purely empirical, since whether you accept P3 depends on your views on (e.g.) what these facts imply about the degree of imprecision of actions’ “EVs”. ↩︎

5. There’s an analogy to the philosophy literature on skepticism, and Agrippa’s trilemma: Even if you think the global skeptic is obviously wrong, it matters a lot whether your alternative to global skepticism is foundationalism, coherentism, or infinitism. ↩︎

6. See also Violet Hour’s call for forecasting generalizability research. ↩︎

7. Quote: “But from another perspective, every decision in life involves a “bet” of sorts on which action to take. The best available action may involve keeping your options open, delaying decisions, and gathering more information. But even those choices are still “part of the bet”. At the end of the day, you still have to choose an action. Humans can’t generate precise credences. … But when it comes time to act, we still have to cash out our uncertainty.” ↩︎

8. Quote: “Rejecting premise 1, completeness is essentially a nonstarter in the context of morality, where the whole project is premised on figuring out which worlds, actions, beliefs, rules, etc., are better than or equivalent to others. You can deny this your heart of hearts - I won’t say that you literally cannot believe that two things are fundamentally incomparable - but I will say that the world never accommodates your sincerely held belief or conscientious objector petition when it confronts you with the choice to take option A, option B, or perhaps coin flip between them.” ↩︎

9. Someone could agree that an appeal to heuristics by itself can’t justify a c-preference, but argue that some heuristic does indeed track the “expected” consequences. The sequence addresses this argument in Sec. 4.1.4. ↩︎

10. Quote: “My principal interest is the pragmatic one: that agents like ourselves make better decisions by attempting to EV-maximization with precisification than they would with imprecise approaches.” ↩︎

11. Quote: “If the argument from cluelessness depends on giving that kind of special status to imprecise credences, then I just reject them for the general reason that coarsening credences leads to worse decisions and predictions.” ↩︎

12. For example, suppose the standard is: “Compare how much utility we achieve on average over a set of decision problems, when we follow different procedures (one of which is ‘adopt some precise credences, then explicitly maximize EV’). The best decision is one that adheres to the best-performing procedure, by this metric.” This reduces to: “The best decision is one that adheres to a procedure that maximizes utility in expectation over some precise distribution over past decision problems.” One could give an independent motivation for privileging such a distribution over decision problems — in particular, argue that our beliefs about our current decision problem should precisely match the frequencies of problems in some reference class. But then we’re just back to debating the merits of precise beliefs themselves. ↩︎

13. As discussed in “How to not do decision theory backwards”, section “Objections and responses”, this view doesn’t assume a foundationalist view of justification. Nor does it deny that all intuitions can provide defeasible justification. ↩︎

14. Quote: “I think most of us feel like we’re really just making up arbitrary numbers, but that’s really uncomfortable because precisely which arbitrary numbers we make up seems to make a difference to what we ended up doing.” See also Greaves’s discussion of the “decision discomfort” involved in complex cluelessness. ↩︎

15. Quote: “Now, I agree that this scenario is ridiculous. And that it sucks. And I agree that picking a precise minute feels uncomfortable. And I agree that this is demanding way more precision than you are able to generate. But if you find yourself in the game, you’d best pick the minute as well as you can. When the gun is pressed against your temple, you cash out your credences.” ↩︎

16. If this claim is about normative standards, why do I classify it as a critique of the conceptual premise? Because I think the root of the critique is a conceptual misunderstanding, namely, of the structure of the arguments against precision. ↩︎

17. I haven’t seen anyone make “better than chance” precise, but this seems to me to be what people have in mind when they say this. ↩︎

18. Quote: “In the same way our track record of better-than-chance performance warrants us to believe our guesses on hard geopolitical forecasts, it also warrants us to believe a similar cognitive process will give ‘better than nothing’ guesses on which actions tend to be better than others, as the challenges are similar between both.” ↩︎

19. If we claim that we have better-than-chance intuitions about how to weigh up (i) vs. (ii), the same problem recurs. In particular, it remains ambiguous how to weigh up (i) vs. (ii) after updating on our higher-order intuition. ↩︎

20. Quote: “Then there's really philosophical cluelessness, where you change who gets born in the coming thousands of years. On that, I'm pretty happy with the standard Bayesian response, which is: yes, any of your actions have some large chance of doing harm through these weird butterfly effects, but the chance of harm cancels out against the chance of actually doing even more good than you expected. So you end up going back to looking at the things you actually can estimate.” ↩︎

21. Quote: “And if I expect that I have absolutely no idea what the black swans will look like but also have no reason to believe black swans will make this event any more or less likely, then even though I won't adjust my credence further, I can still increase the variance of my distribution over my future credence for this event.” ↩︎