This competition entry has been selected for publication by the Forum team.
TLDR
We propose a rule-of-thumb for handling our cluelessness, what we dub the “cosmic house-always-wins” rule, after the famous gambler’s saying, which advises that we estimate the expected value that we are clueless about by assuming that powerful rational agents will seek to control events in their favor; we explain the necessity of the rule, derive it from expected value theory, expand it to cosmic scales, discuss the consequences, and respond to some possible pushback. (Our argument will also respond to DiGiovanni’s “Summary” argument for cluelessness, serving as an attempted critique of DiGiovanni’s Premise 2b, or should the critique fail, a constructive proposal for dealing with cluelessness.)
Prelude
“Whatever uncertainty besets these processes must necessarily extend to all our reasonings about happiness. I have no wish to exaggerate these uncertainties, feeling that we must all continue to seek happiness for ourselves and for others, in whatever obscurity we may have to grope after it: but there is nothing gained by underrating them, and it is idle to argue as if they did not exist.” ~Henry Sidgwick
“What is the point? We assume that every time we do anything we know what the consequences will be, i.e., more or less what we intend them to be. This is not only not always correct. It is wildly, crazily, stupidly, cross-eyed-blithering-insectly wrong!” ~Douglas Adams
Considering Cluelessness
Here we will address the problem of cluelessness for expected value theories and a path towards a solution.
Recognizing our Cluelessness
So, we have recognized that we are clueless:
- The Assumption: Expected value Theory (EV), the decision theory which prescribes choosing the option with the highest expected value amongst various options of uncertain value, guides the actions of rational agents (Neumann and Morgenstern). (This is tantamount to accepting DiGiovanni’s Premise 1 in his unawareness sequence).
- The Problem: expected value theories can be action-guiding given uncertainties, by assigning probabilities and values to different options; but expected value theories cannot be action-guiding given a clueless remainder, the indeterminate expected value of options that we are truly clueless about (Tarsney et al., Greaves, DiGiovanni, “Unawareness”).
- The Question: So, can any plausible general rule-of-thumb successfully systematically compensate for our cluelessness, one way or the another, and re-render expected value theory action-guiding?
The Trouble with Inductive Strategies
Worryingly, it doesn’t seem like normal scientific method will do to resolve the problem of the clueless remainder because there will always be some further clueless remainder:
- The Inductive Trouble: The trouble is that we could exhaustively empirically eliminate sources of cluelessness, but as long as there is just one leftover clueless remainder, then we are still clueless—for a historically rich version of this problem, see the “problem of induction” (Hume, Henderson). So, a robust solution to cluelessness cannot be derived from such an exhaustive empirical elimination strategy (DiGiovanni, “Unawareness”).
- An Analogy: We can investigate every nook and cranny and eliminate every single bacterium from the area, but if just one remains outside our knowledge it could cause a population explosion, such that we have to start cleaning all over again.
Putting the inductive trouble in a formal argument:
The Argument Against Piecemeal Eliminative Inductive Strategies regarding Cluelessness:
- We can exhaustively empirically determine X% of all facts.
- But, even if we exhaustively empirically determine X% of all facts, there still could be 1-X% of facts leftover that are indeterminate (Hume, Henderson).
- Any Y% of facts may be extremely important.
- Therefore, there is always still some Z% of extremely important facts that still could still be indeterminate.
Because of these considerations, we may want to move away from inductive solutions and towards a deductive solution to cluelessness instead. (This is tantamount to accepting DiGiovanni’s Premise 3, in his “Summary” argument, the empirical premise, in his unawareness sequence).
Towards some Deductive Strategies
So, when induction fails, deduction may be our only hope:
- A Deductive Solution: Rather than being empirically exhaustive, a robust methodology against the clueless remainder seemingly must involve the strictly deductive derivation of a better-than-guess strategies for assigning expected value to our clueless remainder from first principles.
Putting the deductive solution in an formal argument:
The Argument For Deductive EV-Based Strategies regarding Cluelessness
- Either estimate the clueless remainder with inductive or deductive strategies based on our first principles.
- Inductive strategies are always incomplete (see previous argument).
- Occam's Razor: assume only what is necessary (Ockham).
- Von Neumann–Morgenstern Theorem: rational agents are already assuming expected value theory (Neumann and Morgenstern).
- Therefore, estimate the clueless remainder using a deductive strategy that assumes expected value theory and little else.
But could any such strategies even exist? The rest of this work will propose and defend a candidate for a deductive EV-based strategy for solving cluelessness. (This is tantamount to questioning DiGiovanni’s Premise 2, in his “Summary” argument, the conceptual premise, in his unawareness sequence. Particularly, we will be focusing on Premise 2b, the modest version of the premise that attempts no more than better-than-guess.)
The House Always Wins Rule
Here we will propose that the so-called “house-always-wins” rule is one candidate for a deductively derived EV-based strategy to cluelessness.
Mystery Box Casino Example
So, our question is: what does a deductively derived EV-based strategy for estimating the clueless remainder look like? To approach answering, let’s turn to one such candidate strategy, illustrating it by examples.
Mystery Box Casino Example:
- Option A: when you pay 100$ to roll a die; if you roll 1, you win 400$; if you roll 2-6, you win nothing; total EVA = − (5/6)(100) + (1/6)(300) =− (200/6)
- Option B: when you pay 100$ you win a mystery prize, about which you are clueless; total EVB = EVmysterybox − 100 , where EVmysterybox =?
- Option C: you leave the casino; total EVC = 0
About Option A, we can say we are uncertain, because we can assign probabilities to the dice rolls; but of the mystery box of Option B, we must say we are clueless, because its expected value is indeterminate, because there is no way to assign probabilities to it like we can with the dice of Option A; the mystery box is statistically opaque. So, given cluelessness, we cannot make an expected value comparison.
But is this strictly true? Perhaps not, because notably, usually in such cases we think it is rational to choose Option C, in spite of the indeterminacy of Option B, because we have adopted a “house-always-wins” rule-of-thumb.
Proposing a “House-Always-Wins” Rule
At least in the narrow case of betting odds in a casino, we know of at least one rule-of-thumb by which to trivially dissolve the clueless remainder:
The House-Always-Wins Rule (HAW): You may assign values to your clueless remainder consistent with a payout matrix that advantages the agents that seek to control the payout matrix.
We can attempt to apply the HAW to our previous mystery box example to illustrate how it locally dissolves the problem of cluelessness.*
Example:
- Option B: when you pay 100$ you win a mystery prize, about which you are clueless, but using the HAW, is assumed to be of a value consistent with the house advantage; total EVB = EVmysterybox − 100 , where EVmysterybox = X(EVhouse ≥ 0)
We can thereby estimate the range of X:**
EVhouse = 100 − X ≥ 0
100 ≥ X
EVB < 100 − 100 = 0
So, with Option A and Option B both having negative EV, this means Option C, at zero EV, is the best guess according to the HAW.***
*We also attempt to empirically assess the expected value of the mystery box. We could do this by tallying up some large sample of previous prizes, running some statistical analysis, and then predicting future boxes. However, this would be both futile and overcomplicated: futile because even if we could get good data and conduct this rigorous analysis, the house could switch up the rules at the last minute, and so we would still be clueless; overcomplicated because we routinely accurately make these kinds of judgements without doing any such calculations, by appeal to the HAW.
**Notably, the HAW does not give us a precise EV, just a better-than-guess range of EVs that can be compared to other better-than-guess ranges of EVs.
***Obviously, we can set up other games with other payout structures, but we suggest that any realistic casino game will collapse to some uncertainty/cluelessness combination with uncertainty/cluelessness in house favor. Even uncertainty that favors the player should be deemed suspicious because there may be some costs/risks we are unaware of that shift the odds back into the house’s favor (loaded dice, hidden fees, etc.).
Bounding the House-Always-Wins Rule
The HAW as we have written, describes a range of reasonable values to assign our cluelessness, but it can also be described by the two bounds of that range:*
- Optimistic HAW Bound (OHAWB): At best, you may assign values to your clueless remainder consistent with a payout matrix in which the controlling agents are attempting to break even on their EV.
- Pessimistic HAW Bound (PHAWB): At worst, you may assign values to your clueless remainder consistent with a payout matrix in which the controlling agents are attempting to maximize their own EV.
This gives us a better-than-guess range of EVs to compare to other better-than-guess ranges of EVs.
*Notably, whereas the OHAWB is a tight bound, the PHAWB is a loose bound: downside is worse than upside is better.
Why might the HAW Work against Cluelessness?
The reason the HAW might work in spite of and against cluelessness is because it is a better-than-guess rule derived from the assumption of expected value theory itself.
The Expected Value Argument for HAW:
- a) The house is attempting to increase its own EV.
- The payout matrix is sought to be controlled by the house.
- Therefore, any indeterminacy in the payout matrix is sought to be constrained to the domain that increases the house’s EV.
- b) We are attempting to increase our own EV.
- Therefore, we should choose as though any indeterminacy in the payout matrix is sought to be constrained to the domain that increases the house’s EV.
This argument shows that the HAW emerges from expected value theory (Premise 1a and 1b) and a local control assumption (Premise 2). The control assumption itself needs examination and explanation though.
Why Should we Believe the Control Assumption holds?
So, why should we accept the assumption of control (Premise 2)? The control assumption can follow analytically from the expected value theory itself, given instrumental rationality.
The Expected Value Argument for Cosmic Control:
- Expected Value Theory: The rational agent will seek to increase its own expected value (von Neumann and Morgenstern).
- Instrumental Rationality: Increasing control (the means for increasing expected value) increases expected value (Bostrom and others).
- Therefore, the rational agent will seek to increase its control.
The instrumental rationality thesis (Premise 2) has been adopted by various thinkers (Bostrom and others) and seems to be definitional given the existence of some things as means of obtaining expected value.
What does the HAW Advise against Cluelessness?
The HAW advises:
- Assume the clueless remainder leans towards the expected advantage of the house.
- Exclude options that increase your expected value at the expected expense of the house.
Applied example:
Gambling Aversion: We know that playing any game against the casino pits your expected value against the expected value of the casino with our better-than-guess assessment of the clueless remainder leaning in favor of the casino and against you, so such games should be avoided.
The Cosmic House-Always-Wins Rule
Here we will extend the house-always-wins rule from the local domain of casinos to the cosmos of all domains controlled by rational agents.
Can there be a “Cosmic House-Always-Wins” Rule?
To approach a generalizable HAW, first, we need to recognize the narrowness of the HAW, but second, we need to recognize the potential for broadness of the HAW:
- The Traditional Specificity of HAW: As normally construed, the HAW is narrowly context specific, since it specifically applies to casinos, situations where the payout matrix has been carefully set-up by a “house”, a controlling rational agent.
- The Potential Universality of HAW (Cosmic HAW, or CHAW): However, even outside of a casino, the HAW may still generally apply, to the extent that the cosmos resembles a casino, a situation where payout matrices lean towards being set-up by a “cosmic house”, the cosmic set of powerful rational agents.
Why might the CHAW Work against Cluelessness?
The move from the HAW to the CHAW only depends on the extent to which premise 2 holds for the cosmic casino. So, we may keep the above argument for the HAW exactly the same with the addition of the word “cosmic”:
The Expected Value Argument for CHAW:
- a) The (cosmic) house is attempting to increase its own expected value .
- The (cosmic) payout matrix is sought to be controlled by the (cosmic) house.
- Therefore, any indeterminacy in the (cosmic) payout matrix is sought to be constrained to the domain that increases the (cosmic) house’s expected value .
- b) We are attempting to increase our own expected value .
- Therefore, we should choose as though any indeterminacy in the (cosmic) payout matrix is sought to be constrained to the domain that increases the (cosmic) house’s expected value.
This argument shows that the CHAW emerges from expected value theory (Premise 1a and 1b) and a cosmic control assumption (Premise 2). The cosmic control assumption is much broader than the local control assumption of a casino, so it needs examination and explanation: the move from seeking local to seeking cosmic control follows from the instrumental rationality thesis (discussed above), because there is no local condition placed on instrumental rationality. Extrapolating this into the long-term, we can adopt the prediction that the future will be sought to become more rationally controlled than the past, to the extent possible, in the limit case leading towards futures where rational control gradually swamps out irrational chance in all controllable domains.
What does a CHAW advise against Cluelessness?
The CHAW advises:
- Assume clueless remainder leans towards the expected advantage of the “cosmic house” (the cosmically powerful rational agents).
- Exclude options that increase your expected value at the expected expense of the cosmic house.
An applied example:
Geopolitical Conformism: The powerful agent on Earth is the global superpower (perhaps estimated as the balance of power amongst the alliances of nations), so an average earthling should make better-than-guess assessments of the clueless remainder consistent with the expected interests of that superpower and do work accordingly.
What if the CHAW itself is Uncertain?
But what do we do if we are uncertain about the nature of the Cosmic House? The Cosmic House may have unusual features:
- Distributed: lots of agents may have substantial power.
- Reflective: powerful agents may adjust their expected value assessments based on their perception of the expected value assessments of other agents.
- Uncertain: the power of any given agent may be unclear.
Therefore, we can propose a special version of the CHAW:
A Distributed, Reflective, Uncertain CHAW: Given uncertainty about who, if anyone is the most powerful agent, and what they think of other agents, we should anticipate the House expected value to be an aggregative function of uncertainly reflective distribution across the population of affecting agents, and assign uncertain ranges of values to our clueless remainder accordingly.
The DRUCHAW may more plausibly broadly apply to larger scale, distributed, uncertain situations. It also has the interesting effect of, given cluelessness, diluting any given expected value into an uncertainly reflected distributed expected value (at the egalitarian limit approaching impartial altruism).
An applied example:
The Storm Planet of The Rats: imagine a universe composed of a single solar system with a single habitable planet, on which lives a species called the Rats. The planet is extremely stormy, so stormy in fact that it remains largely unpredictable in spite of the finest weather models. However some of the Rats are instrumentally rational and seek control over their environment.*
What should we expect to become of the expected value of such a planet in the long-term?
- According to Cluelessness: we cannot make any determinate predictions because chaotic weather dominates our expected value comparisons, and no matter how meticulously well- researched our predictions become, we can never rule out the indeterminate possibility of extinction-level storms.
- According to CHAW Expected Value Theory: we can make the CHAW better-than-guess prediction that, because some of the Rats are rational, and because rational creatures seek to control their environment, the rational Rats will gradually seek to whittle away their cluelessness about their environment, albeit with some remaining uncertainty about their competency.
*There may be something interesting to say here about the gradual natural selection of rational agents, but we will pass over those evolutionary considerations for now.
The Upshot of the CHAW?
Why does all this matter? The upshot is that the CHAW offers a generalizable EV-based better-than-guess workaround strategy for cluelessness. There are two ways of interpreting the strategy:
- A Critique: we can construe the CHAW as a conceptual critique of arguments for cluelessness that offers a better-than-guess expected value comparison. Some worry that we must refine the course-grainedness of our understanding to make rational expected value comparisons. For example, DiGiovanni says:
If our understanding of A’s and B’s possible consequences is sufficiently coarse-grained, then we don’t have an argument for “expecting” our idealized self’s EV for A to be higher, lower, or equal to B’s. So A’s and B’s “EVs” are incomparable. (DiGiovanni, “Summary”, Premise 2).
But if we accept CHAW, we may push back against this, because CHAW suggests that we can still be rationally expecting and comparing without further refining our course-grained understanding by estimating the clueless remainder as consistent with the expected value of a theoretical rationally controlling “house”. (In particular the CHAW offers a better-than-guess EV range, not a precise EV, which specifically challenges DiGiovanni’s Premise 2b, while basically agreeing with DiGiovanni’s Premise 2a.)
- A Proposal: Even if we think this conceptual critique fails (if the above argument has made some formal mistake), we can still construe the CHAW as a constructive proposal in the face of clueless: yes, we are formally clueless, but we still must act, so we can adopt CHAW as a workaround strategy, the upshot being that we can perhaps rule out certain kinds of actions as according to CHAW-modified expected value theory.
Either way, what the CHAW attempts to allow expected value theorists to do is make reasonable near-term expected value better-than-guess comparisons, and sidebar the long-term expected values, by assuming that long-term knock-on effects will be adjusted, re-adjusted, and re-re-adjusted by rational agents nudging the chaos of the world into more alignment with rational control.
Interlude
“Remember then: there is only one time that is important— Now! It is the most important time because it is the only time when we have any power. The most necessary man is he with whom you are, for no man knows whether he will ever have dealings with any one else: and the most important affair is, to do him good, because for that purpose alone was man sent into this life!” ~Leo Tolstoy
“The gods, likening themselves to all kinds of strangers, go in various disguises from city to city, observing the wrongdoing and the righteousness of men.” ~Homer
Some Consequences of the CHAW Rule
Here we will assess some of the most salient consequences of adopting CHAW rule.
Some General Intuitions Consistent with CHAW
The CHAW may be prima facie consistent with other general rational intuitions:
- Win-Win: because the house may seek to hopelessly rig the game, if possible, default towards win-win strategies.
- Risk-Aversion: because the PHAWB is looser than the OHAWB, if possible, default towards risk-aversion.
- Knowledge of Power: because the expected value and power of the “house” must be understood, even with substantial uncertainty, knowledge of the dynamics of power should be acquired, and therefore one of the primary cause areas for the expected value theorist should be an understanding of geopolitics.
However, our precise conclusions about such matters may require more analysis to clearly explore.
Some Special Limit Cases for the CHAW
There are a few special limit cases worth considering when thinking about the CHAW:
- Big Brother: if we have high credence that an all-powerful godlike surveillance system is fully controlling our environment according to its expected value (an obvious cosmic house), then we can roughly treat our clueless remainder as within the range of the all-powerful god’s expected value (Orwell).
- Robinson Crusoe: if we find ourselves alone, in the absence of any recognizable “cosmic house”, then we can roughly treat ourselves as the cosmic house and can roughly treat the clueless remainder as consistent with our own control of our own expected value (Defoe).
- Everything In Between: in intermediate cases, where we are neither fully alone nor fully surveilled, we can roughly treat the clueless remainder as consistent with the an uncertain distributed aggregate of the EVs of the dominant agents.
In these limit cases, the CHAW still applies and can handle them.
The CHAW rule relates to other moral principles:
- Citizenship & Stewardship: because people usually find themselves in the presence of powerful institutions, the CHAW leans towards good citizenship; and because institutions find themselves potentially overruled by the people, the CHAW presses towards good stewardship (Hobbes).
- Theo-Xenia: because we may obliviously find ourselves in the presence of gods in disguise, who may compose or represent the cosmic house, who we do not want to upset, the CHAW presses towards win-win relationships with strangers (Homer).
- Impartial Altruism: because, if perfect equality obtains, the House expected value would be identical with the overall total expected value of the population, the CHAW presses towards uncertainly distributed altruism, and impartial altruism in cases of equality (Tolstoy).
The CHAW can help explain these moral principles on an expected value basis.
Some Classic Cause Areas Consistent with CHAW
The CHAW rule is consistent with a range of cause areas:
- Earning to Give: earning to give is permissible as long as the earning and the giving are win-win for the global community. For example, becoming a well-paid doctor and donating to win-win charities may be permissible; but, that same doctor donating to fund win-lose conflicts may be discouraged; likewise, becoming a well-paid assassin and donating to win-win charities may be discouraged.
- Global Health: global health initiatives are permissible as long as the distribution of health outcomes are win-win for geopolitics. For example, distributing aid to a wide range of needy regions may be permissible; distributing aid disproportionately to one side of a two-sided geopolitical conflict may be discouraged.
- Animal Welfare: animal welfare initiatives are permissible as long as the improvements of the lives of the animals are consistent with the improvements of the lives of the humans engaging in animal husbandry. For example, increasing animal welfare at zero cost to humans may be permissible; whereas, increasing animal welfare to the detriment of humans may be discouraged.
- X-Risk Mitigation: extinction risk mitigation initiatives are permissible as long as the mitigations are not themselves likely catastrophic. For example, unilateral nuclear reduction may be permissible; preemptive nuclear strikes may be discouraged.
In these classic cause areas and others, the CHAW offers a range of permissible actions, but with ranges bounded by win-win dynamics.
Superintelligence as the Ultimate Cosmic House Advantage?
One implication of the CHAW is that we should avoid conflict with powerful agents, especially super powerful agents, like superintelligences:
The CHAW argument against adversarial encounters with rational superintelligence:
- The CHAW can lead us to systematically avoid options leading us into high-cost, zero-sum, rigged casinos.
- To an extent, adversarial encounter with rational superintelligence would resemble entering a high-cost, zero-sum, rigged casino.
- Therefore, to an extent, the CHAW can lead us to systematically avoid options leading towards adversarial encounters with rational superintelligence.
This argument leads us to adopt precaution according to CHAW when considering cases involving superintelligence.
Some Cosmic Acts CHAW May Prima Facie Precaution
So, the CHAW may prima facie bias our expected value assignments against certain cosmic actions like encounters with superintelligence:
- Against ASI Development: if powerful ASI is possible, CHAW may caution against haphazard development (Bostrom).
- Against ETI Messaging: if powerful ETI exists, CHAW may caution against frivolous contact (Brin).
- Against Sim-Tampering: if we are in a simulation, CHAW may caution against attempting to alter or exit the programming (Chalmers).
However, precise conclusions about how to handle such encounters may require more analysis to clearly explore.
A CHAW Corollary regarding Superintelligent Alignment
As a corollary, another implication of CHAW can give us slight hope for solving the alignment problem:
The Argument for Clueless Rational Alignment:
- All clueless rational agents must respect CHAW.
- Therefore, clueless rational superintelligence must respect CHAW. (via Universal Instantiation)
Although we may be uncertain about this corollary, the upshot would be a roadmap towards easy alignment through “clueless rationality”, which may be consistent with certain alignment programs already being pursued (Russell).
Objections and Responses
Here we will address the most salient objections to the reasoning that brings us the CHAW, and we will make some responses.
Some Possible Objections to HAW
- The HAW regards uncertainty, not cluelessness.
- Yes, cases of calculable expected value merely involve uncertainty (like rolling dice), but other cases (like the mystery box case) involve cluelessness because the expected payout is wholly indeterminate (I have zero info on how to evaluate the expected value of the mystery box).
- The HAW is also empirical, not analytic:
- Certainly there is an empirical component when determining the relative levels of control in identifying the house, but the rule as derived above is based purely in expected value theory without further empirical assumptions.
- The HAW does not follow from expected value theory.
- Well, according to expected value theory, if we are attempting to increase our EV, and doing so depends on accurately modeling other rational agents, we should model other rational agents as though they are attempting to increase their expected value as well, which seems to imply the HAW. (See von Neumann’s theorem.)
- The HAW control assumption does not follow from expected value theory.
- On the one hand, we may believe that the control assumption is empirically false and that we have smuggled in empirical assumptions into our analysis. But this is not intended. Rather, expected value theory is advising that we act as if the control assumption were true, regardless of our uncertainty about control, because it is consistent with our expected value theory assumption.
- One the other hand, we may believe that the control assumption is analytically false. However, this seems untenable, since by control we simply mean the means for increasing expected value, and we are assuming that rational agents generally seek control of their expected value, which seems to make the control assumption to definitionally follow from expected value theory.
- The HAW is a rule amongst egoists, not impartial altruists.
- Okay, but even impartial altruists are attempting to increase their personal expected value function (it just happens to be the impartial altruistic expected value function), and so they too can be modeled like other agents with expected values. (See von Neumann’s theorem.) And we may take it that if the house happens to be an impartial altruist whose expected value is aligned with ours, then in a strong sense we just lucked into best case: a built in failsafe of the HAW.
- The HAW can lead to infinite reflective interpersonal regress.
- Fair enough, if all rational agents adopt the HAW, then many agents may reflect upon the behavior of other agents, leading them to adjust their expected value assessments based on what they think are the expected value assessments of other rational agents, leading to possible infinite regress. However, this regress should lead naturally towards convergence, not divergence, and reflective equilibrium, assuming that these agents have converging epistemological methods (expected value theory).
Some Possible Objections to CHAW
- The HAW is not a good cosmic rule because the universe is not a cosmic casino run by a “house”.
- Fair enough, the universe is not explicitly a casino, but we still believe we can apply the HAW at many scales. We apply it to behavior in casinos. We also apply a version of it to behavior as citizens of communities, cities, and states. We even apply it to the geopolitical order. The only extra step being made is to generalize it, not necessarily by extrapolation, but by stipulation of the expected behavior of rational agents according to expected value theory.
- The HAW is not a good cosmic rule because the universe is run by many “houses”.
- Fair enough, there are many powerful rational agents, but the expected value of a theorized “cosmic house” may still be approximated as an aggregative function of the dominant set of aligned agents amongst the sets of adversarial agents composing the cosmos. Thus, CHAW becomes a social consideration of approximating the aggregate expected value function amidst uncertainty (see DRUCHAW above).
- The HAW is not a good cosmic rule because the cosmic house is not necessarily rational.
- Sure, the agents in the cosmos may be irrational and chaotic, however, if we accept the instrumental rationality thesis, we should also expect the rational agents that exist to seek control, and controlling agents preside, thereby becoming the theoretical “cosmic house”, since we are defining the cosmic house as the set of rational agents seeking control. (See next objection for adjacent considerations.)
- The HAW is not a good cosmic rule because there is clueless remainder even after the domain controlled by rational agents is considered.
- Agreed, this is a big worry. The defense here is that according to our derivation of CHAW, the expected house advantage is assumed to seek to swamp out the clueless remainder per the control assumption. In other words, there is no long-term clueless domain outside of what is sought to be expected house advantage because we have assumed expected value theory and assumed that based on expected value theory we should expect rational agents seek to control the clueless remainder.
- Furthermore, even if rational control fails and irrational chance prevails, utterly, which it still could, given cosmic uncertainty, then our CHAW estimates will have been proved inaccurate, but not irrational. They are still rational because, rationally assuming expected value theory, it is rational to expect that rational agents will seek to control the contingencies of the clueless remainder, as those rational agents gain control over the universe, rational agency swamping out irrational contingency.
Some Possible Objections to Applying CHAW
- The CHAW works, but superintelligence is not like the house.
- Sure, maybe the superintelligence is something unexpected and irrational. But we are merely stipulating that a rational superintelligence is a hyper-house? (See von Neumann’s theorem.)
- The CHAW works, but it does not make such specific conclusions about earning to give, global health, animal rights, and x-risk.
- Fair, applying the theory here is what we have explored in the least depth in this analysis, and we can be open to our minds being changed about various cause areas, and this is an area of further research. Though we may feel strong prima facie CHAW advocacy for effective charities and strong risk-aversion to x-risks.
Some Further Worries Regarding CHAW
- The CHAW is just a version of Pascal’s Wager (Pascal):
- Fair, though perhaps this gets it backwards: Pascal’s Wager is a specific version of the CHAW. In as much, Pascal’s Wager assumes more about the nature of the theoretical “cosmic house” (for Pascal, the Christian God) than the generic CHAW does. In other words, the CHAW is open to updates about the nature of the house.
- The CHAW seems fairly conservative and conformist (Burke).
- Fair, but we can bite this bullet and say, yes, probably, since the downside seems deeper (PHAWB) than the upside (OHAWB). But some long-term epistemic conservatism and/or conformism may be the right attitude towards extremely uncertain and a precarious universe.
- Also, CHAW conservatism is only situational, a conditional rule, but CHAW can also be extremely liberal and progressive at times as well. This is because the CHAW is itself agnostic about the nature of the house, without further updating, and may therefore favor conservative or progressive action depending on the balance of powers in any given situation.
- The CHAW seems fairly modest (Bommarito).
- Fair, the CHAW mostly advises against certain horrible win-lose options, but it does not strongly favor specific cause areas. This may be a blessing in disguise though: we mostly know what not to do, but this gives us wide range of free play, with lots of win-win options to choose amongst.
- CHAW modesty also does not overstate our epistemic situation.
- The CHAW holds, but we are clueless about the nature of the house.
- Fair, this should be taken very seriously because it certainly seems like if we are serious about cluelessness, we could apply cluelessness back to the solutions to cluelessness. If we are clueless about all solutions to cluelessness, then we are still clueless. However, if we are accepting expected value theory anyway, as the basis for the CHAW, then we are already accepting the totalizing approach by which we developed the CHAW. So, we are only as clueless about CHAW as we are about our expected value theory assumptions. So, we should only reject CHAW on clueless grounds if we reject our expected value theory assumptions on clueless grounds. If this is right, to any clueless remainder we have regarding the nature of the house, we may simply reapply the CHAW back to clueless remainder again, a virtuous circle in which the clueless remainder is always estimable as consistent with the CHAW, towards a vanishing point.
- The CHAW holds, but we are uncertain about the nature of the house.
- Fully agreed. Even though the house is seeking rational control, they are still subject to incompetency and so we are subject to uncertainty. Here we may propose that the further project for the expected value theorist who accepts the CHAW is to engage in some best-guess reasoning about the ultimate expected value and power of the structures governing the world as we know it. This makes estimating our clueless remainder the same thing as estimating our uncertainty about the competency of the agents in control. This is not a trivial task itself, because it involves a complicated consideration of the global game theoretic dynamics of power. But this turns the abstract project of cluelessness into an actionable study of geopolitics.
A Personal Aside
Breaking the fourth wall, what is my actual personal credence that any of this is true? Not sure. But, upon self-reflection, something like the CHAW does seem to be how I behave: I do what seems like high near-term expected value, and I assume that the long-term will be refined by future, better people, those with the power to adjust things as necessary as they come. So being able to reverse-engineer my default behavior from expected value theory does boost my confidence. My primary source of doubt has been flagged above: sometimes I pessimistically wonder whether the control assumption is actually sufficient to accurately encompass the clueless remainder in the long-term arc of cosmic history; other times though, I think that proving the accuracy of the rule is unnecessary, and all we need is a rule consistent with some version of expected value theory, like the CHAW, to justify our actions rationally. I just hope the resulting strategy is marginally more tractable than cluelessly assigning EVs, which I think it may be, because it assigns modest ranges to rational behavior without expecting more.
Summary and Conclusion
We have argued that the problem of cluelessness for expected value theory should be approached by leaning away from win-lose engagements, a house-always-wins rule; and that, given some modest assumptions, this can be scaled up into a cosmic house-always-wins rule to help us make important global decisions; and that this cosmic rule-of-thumb gives answers that are consistent with many other intuitions, moral principles, and traditional cause areas. We suggest that this offers a modest workaround for the formal cluelessness problem so that the hard work of managing global priorities can proceed.
Acknowledgements
Victoria Tang and Brody McManus for beta-reading and discussing.
References
Adams, Douglas. The Hitchhiker's Guide to the Galaxy: The Trilogy of Five.
Bommarito, Nicolas. "Modesty and Humility". The Stanford Encyclopedia of Philosophy. (Winter 2018 Edition), Edward N. Zalta (ed.). https://plato.stanford.edu/archives/win2018/entries/modesty-humility
Bostrom, Nick. “The Superintelligent Will: Motivation and Instrumental Rationality in Advanced Artificial Agents”. https://nickbostrom.com/superintelligentwill.pdf
Brin, David Brin. “Shouting at the Cosmos”. Lifeboat Foundation. https://lifeboat.com/ex/shouting.at.the.cosmos
Burke, Edmund. The Works of Edmund Burke. https://www.gutenberg.org/ebooks/author/842
Chalmers, David. Reality+: Virtual Worlds and the Problems of Philosophy. https://consc.net/reality/
Defoe, Daniel. Robinson Crusoe.
DiGiovanni, Anthony. “Cluelessness: Summary of the argument, why it matters, and counterarguments.” https://forum.effectivealtruism.org/posts/NesgdEY6yPrE9wDap/cluelessness-summary-of-the-argument-why-it-matters-and-1
DiGiovanni, Anthony. “The challenge of unawareness for impartial altruist action guidance”. https://forum.effectivealtruism.org/s/rHqdsrieinyhM5KDv
Greaves, Hilary, 2016, “Cluelessness”, Proceedings of the Aristotelian Society, 116(3): 311–339. doi:10.1093/arisoc/aow018
Henderson, Leah. "The Problem of Induction". Stanford Encyclopedia of Philosophy. (22 November 2022). Zalta, Edward N.; Nodelman, Uri (eds.). https://plato.stanford.edu/entries/induction-problem/
Hobbes, Thomas. Leviathan. https://www.gutenberg.org/files/3207/3207-h/3207-h
Homer. The Odyssey. https://www.gutenberg.org/cache/epub/1727/pg1727-images.html
Hume, David. An Enquiry Concerning Human Understanding
McAskill, Bykvist, Ord. Moral Uncertainty. https://www.moraluncertainty.com/
Neumann, John von and Morgenstern, Oskar, Theory of Games and Economic Behavior. Princeton, NJ. Princeton University Press, 1953.
Ockham, William. Quaestiones et decisiones in quattuor libros Sententiarum Petri Lombardi.
Orwell, George. 1984.
Pascal, Blaise. Pensees. https://www.gutenberg.org/files/18269/18269-h/18269-h.htm
Russell, Stuart. Human Compatible: Artificial Intelligence and the Problem of Control.
Sidgwick, Henry. The Methods of Ethics. https://www.gutenberg.org/cache/epub/46743/pg46743-images.html
Tarsney, Christian, Teruji Thomas, and William MacAskill, "Moral Decision-Making Under Uncertainty", The Stanford Encyclopedia of Philosophy (Spring 2024 Edition), Edward N. Zalta & Uri Nodelman (eds.), URL = <https://plato.stanford.edu/archives/spr2024/entries/moral-decision-uncertainty/>.
Tolstoy, Leo. Three Questions. https://en.wikisource.org/wiki/Twenty-three_Tales/Three_Questions
Tremlett, Toby. Announcing: Cluelessness Critiques Competition. https://forum.effectivealtruism.org/posts/2aooibAEDSdreCqrJ/announcing-cluelessness-critiques-competition