All of Richard Y Chappell🔸's Comments + Replies
You keep describing possible scenarios in which the actual value of averting extinction would be low. Do you not understand the scenario conditions under which the actual value would be high? I'm suggesting that those kinds of scenario (in which astronomically valuable futures are reliably securable so long as we avoid extinction this century) seem reasonably credible to me, which is what grounds my background belief in x-risk mitigation having high expected value.
Edited to add: You may in a sense be talking about "expected value", but insofar as it is bas...
Do you not understand the scenario conditions under which the actual value would be high?
I do. In theory, there could be worlds which i) will actually be astronomically valuable given some efforts, but that ii) will be practically actually neutral without such efforts. In this case, the efforts would be astronomically valuable due to meaningfully increasing the probability of astronomically valuable futures. I just think such worlds are super implausible. I can see many worlds satisfying i), but not i) and ii) simultaneously.
...We need to abstract from that s
I really want to stress the difference between saying something will (definitely) happen versus saying there's a credible chance (>1%) that it will happen. They're very different claims!
Lovelace and Menabrea probably should have regarded their time as disproportionately likely (compared to arbitrary decades) to see continued rapid progress. That's compatible with thinking it overwhelmingly likely (~99%) that they'd soon hit a hurdle.
As a heuristic, ask: if one were, at the end of history, to plot the 100 (or even just the 50) greatest breakthrough perio...
It is true that Lovelace and Menabrea should have assumed a credible chance of rapid progress. Who knows, maybe if they had had the right resources and people, we could have had computers much earlier than we ultimately had.
But when talking about ASI, we are not just talking about rapid progress, we are talking about the most extreme progress imaginable. Extraordinary claims require extraordinary evidence, and so forth. We do not know what breakthroughs ASI requires, nor do we know how far we are from it.
It all comes down to the question of whether the cur...
Thanks for explaining your view!
On the first point: I think we should view ASI as disproportionately likely in decades that already feature (i) recent extraordinary progress in AI capabilities that surprises almost everyone, and (ii) a fair number of experts in the field who appear to take seriously the possibility that continued progress in this vein could soon result in ASI.
I'd then think we should view it as disproportionately unlikely that ASI will either be (a) achieved before any such initial signs of impressive progress, OR (b) achieved centur...
While it might feel to you that AI progress has been rapid in the past decade, most innovations behind it such as neural networks, gradient descent, backpropagation, and the concept of language models are very old innovations. The only major innovation in the past decade is the Transformer architecture from 2017, and almost everything else is just incremental progress and scaling on larger models and datasets. Thus, the pace of AI architecture development is very slow and the idea that a groundbreaking new AGI architecture will surface has a low probabilit...
I'm a bit confused by this response. Are you just saying that high expected value is not sufficient for actual value because we might get unlucky?
Some possible extinction-averting events merely extend life for 1 second, and so provide little value. Other possibilities offer far greater extensions. Obviously the latter possibilities are the ones that ground high expected value estimates.
I think Shulman's points give us reason to think there's a non-negligible chance of averting extinction (extending civilization) for a long time. Pointing out that other possibilities are also possible doesn't undermine this claim.
Distinguish pro tanto vs all-things-considered (or "net") high stakes. The statement is literally true of pro tanto high stakes: the 1% chance of extremely high stakes is by itself, as far as it goes, an expected high stake. But it's possible that this high stake might be outweighed or cancelled out by other sufficiently high stakes among the remaining probability space (hence the subsequent parenthetical about "unless one inverts the high stakes in a way that cancels out...").
The general lesson of my post is that saying "there's a 99% chance there's nothi...
The responses to my comment have provided a real object lesson to me about how a rough throwaway remark (in this case: my attempt to very briefly indicate what my other post was about) can badly distract readers from one's actual point! Perhaps I would have done better to entirely leave out any positive attempt to here describe the content of my other post, and merely offer the negative claim that it wasn't about asserting specific probabilities.
My brief characterization was not especially well optimized for conveying the complex dialectic in the other pos...
On (what I take to be) the key substantive claim of the post:
I think that nontrivial probability assignments to strong and antecedently implausible claims should be supported by extensive argument rather than manufactured probabilities.
There seems room for people to disagree on priors about which claims are "strong and antecedently implausible". For example, I think Carl Shulman offers a reasonably plausible case for existential stability if we survive the next few centuries. By contrast, I find a lot of David's apparent assumptions about which proposition...
Hi David, I'm afraid you might have gotten caught up in a tangent here! The main point of my comment was that your post criticizes me on the basis of a misrepresentation. You claim that my "primary argumentative move is to assign nontrivial probabilities without substantial new evidence," but actually that's false. That's just not what my blog post was about.
In retrospect, I think my attempt to briefly summarize what my post was about was too breezy, and misled many into thinking that its point was trivial. But it really isn't. (In fact, I'd say that my co...
It's not a psychological question. I wrote a blog post offering a philosophical critique of some published academic papers that, it seemed to me, involved an interesting and important error of reasoning. Anyone who thinks my critique goes awry is welcome to comment on it there. But whether my philosophical critique is ultimately correct or not, I don't think that the attempt is aptly described as "personal insult", "ridiculous on [its] face", or "corrosive to productive, charitable discussion". It's literally just doing philosophy.
I'd like it if people read my linked post before passing judgment on it.
The meta-dispute here isn't the most important thing in the world, but for clarity's sake, I think it's worth distinguishing the following questions:
- Does a specific text—Thorstad (2022)—either actually or apparently commit a kind of "best model fallacy", arguing as though establishing Time of Perils hypothesis as unlikely to be true thereby suffices to undermine longtermism?
- Does another specific text—my 'Rule High Stakes In, Not Out'—either actually or apparently have as its "primary argumentative move... to assign nontrivial probabilities without substant
As I see it, I responded entirely reasonably to the actual text of what you wrote. (Maybe what you wrote gave a misleading impression of what you meant or intended; again, I made no claims about the latter.)
Is there a way to mute comment threads? Pursuing this disagreement further seems unlikely to do anyone any good. For what it's worth, I wish you well, and I'm sorry that I wasn't able to provide you with the agreement that you're after.
Honestly, I still think my comment was a good one! I responded to what struck me as the most cruxy claim in your post, explaining why I found it puzzling and confused-seeming. I then offered what I regard as an important corrective to a bad style of thinking that your post might encourage, whatever your intentions. (I made no claims about your intentions.) You're free to view things differently, but I disagree that there is anything "discourteous" about any of this.
There's "understanding" in the weak sense of having the info tokened in a belief-box somewhere, and then there's understanding in the sense of never falling for tempting-but-fallacious inferences like those I discuss in my post.
Have you read the paper I was responding to? I really don't think it's at all "obvious" that all "highly trained moral philosophers" have internalized the point I make in my blog post (that was the whole point of my writing it!), and I offered textual support. For example, Thorstad wrote: "the time of perils hypothesis is probably f...
This sort of "many gods"-style response is precisely what I was referring to with my parenthetical: "unless one inverts the high stakes in a way that cancels out the other high-stakes possibility."
I don't think that dystopian "time of carols" scenarios are remotely as credible as the time of perils hypothesis. If someone disagrees, then certainly resolving that substantive disagreement would be important for making dialectical progress on the question of whether x-risk mitigation is worthwhile or not.
...What makes both arguments instances of the nontrivial pr
Seems like you and the other David T are talking past each other tbh.
Above you reasonably argue the [facetious] "time of carols" hypothesis is not remotely as credible as the time of perils hypothesis. But you also don't assign a specific credence to it, or provide an argument that the "time of carols" is impossible or even <1%[1]
I don't think it would be fair to conclude from this that you don't understand how probability works, and I also don't think that it is reasonable to assume that the probability of the 'time of carols' should be assumed suffici...
The AI bubble popping would be a strong signal that this [capabilities] optimism has been misplaced.
Are you presupposing that good practical reasoning involves (i) trying to picture the most-likely future, and then (ii) doing what would be best in that event (while ignoring other credible possibilities, no matter their higher stakes)?
It would be interesting to read a post where someone tries to explicitly argue for a general principle of ignoring credible risks in order to slightly improve most-probable outcomes. Seems like such a principle would be pretty...
Quick clarification: My target here is not so much people with radically different empirical beliefs (such that they regard vaccines as net-negative), but rather the particular form of status quo bias that I discuss in the original post.
My guess is that for relatively elite audiences (like those who read philosophy blogs), they're unlikely to feel attached to this status quo bias as part of their identity, but their default patterns of thought may lead them to (accidentally, as it were) give it more weight than it deserves. So a bit of heated rhetoric and stigmatization of the thought-pattern in question may help to better inoculate them against it.
(Just a guess though — I could be wrong!)
Interesting post! Re: "how spotlight sizes should be chosen", I think a natural approach is to think about the relative priorities of representatives in a moral parliament. Take the meat eater problem, for example. Suppose you have some mental representatives of human interests, and some representatives of factory farmed animal interests. Then we can ask each representative: "How high a priority is it for you to get your way on whether or not to prevent this child from dying of malaria?" The human representatives will naturally see this as a very high prio...
Funnily enough, the main example that springs to mind is the excessive self-flagellation post-FTX. Many distanced themselves from the community and its optimizing norms/mindset—for understandable reasons, but ones more closely tied to "expressing" (and personal reputation management) than to actually "helping", IMO.
I'd be curious to hear if others think of further candidate examples.
EA Infrastructure Fund or Giving What We Can? For the latter, "our best-guess giving multiplier for [2023-24] was approximately 6x".
Any intellectual community will have (at least implicit) norms surrounding which assumptions / approaches are regarded as:
(i) presumptively correct or eligible to treat as a starting premise for further argument; this is the community "orthodoxy".
(ii) most plausibly mistaken, but reasonable enough to be worth further consideration (i.e. valued critiques, welcomed "heterodoxy")
(iii) too misguided to be worth serious engagement.
It would obviously be a problem for an intellectual community if class (ii) were too narrow. Claims like "dissent isn't welcome" imp...
I'm not seeing the barrier to Person A's thinking there's a 1/1000 chance, conditional on reaching the 50th century, of going extinct in that century. We could easily expect to survive 50 centuries at that rate, and then have the risk consistently decay (halving each century, or something like that) beyond that point, right?
If you instead mean to invoke, say, the 50 millionth century, then I'd think it's crazy on its face to suddenly expect a 1/1000 chance of extinction after surviving so long. That would no longer "seem, on the face of it, credible".
Am I missing something?
Thanks, yeah, I like your point there that "false negatives are costlier than false positives in this case", and so even <50% credence can warrant significant action. (I wouldn't literally say we should "act as if 3H is true" in all respects—as per Nuno's comment, uncertainty may justify some compounding "patient philanthropy", which could have high stakes if the hinge comes later. But that's a minor quibble: I take myself to be broadly in agreement with your larger gist.)
My main puzzlement there is how you could think that you ought to perform an act that you simultaneously ought to hope that you fail to perform, subsequently (and predictably) regret performing, etc. (I assume here that all-things-considered preferences are not cognitively isolated, but have implications for other attitudes like hope and regret.) It seems like there's a kind of incoherence in that combination of attitudes, that undermines the normative authority of the original "ought" claim. We should expect genuinely authoritative oughts to be more wholeheartedly endorsable.
Right, so one crucial clarification is that we're talking about act-inclusive states of affairs, not mere "outcomes" considered in abstraction from how they were brought about. Deontologists certainly don't think that we can get far merely thinking about the latter, but if they assess an action positively then it seems natural enough to take them to be committed to the action's actually being performed (all things considered, including what follows from it). I've written about this more in Deontology and Preferability. A key passage:
...If you think that other
Thanks! You might like my post, 'Axiology, Deontics, and the Telic Question' which suggests a reframing of ethical theory that avoids the common error. (In short: distinguish ideal preferability vs instrumental reasoning / decision theory rather than axiology vs deontics.)
I wonder if it might also help address Mogensen's challenge. Full aggregation seems plausibly true of preferability not just axiology. But then given principles of instrumental rationality linking reasons for preference/desire to reasons for action, it's hard to see how full aggregation c...
Sorry, I don't think I have relevant expertise to assess such empirical claims (which is why I focus more on hypotheticals). It would certainly be convenient if helping people turned out to be the best way to also reduce non-human suffering! And it could be true (I don't take convenience to be an automatic debunker or anything). I just have no idea.
Thanks for your reply! Working backwards...
On your last point, I'm fully on board with strictly decoupling intrinsic vs instrumental questions (see, e.g., my post distinguishing telic vs decision-theoretic questions). Rather, it seems we just have very different views about what telic ends or priorities are plausible. I give ~zero credence to pro-annihilationist views on which it's preferable for the world to end than for any (even broadly utopian) future to obtain that includes severe suffering as a component. Such pro-annihilationist lexicality strikes m...
Regarding the "world-destruction" reductio:
this isn't strong evidence against the underlying truth of suffering-focused views. Consider scenarios where the only options are (1) a thousand people tortured forever with no positive wellbeing whatsoever or (2) painless annihilation of all sentience. Annihilation seems obviously preferable.
I agree that it's obviously true that annihilation is preferable to some outcomes. I understand the objection as being more specific, targeting claims like:
(Ideal): annihilation is ideally desirable in the sense that it...
Isn't the point of the Long Reflection to avoid "locking in" irreversible mistakes? Extinction, for example, is irreversible. But large population isn't. So I don't actually see any sense in which present "min-natalism" maintains more future "optionality" (or better minimizes moral risks) than pro-natalism. Both leave entirely open what future generations choose to do. They just differ in our present population target. And presently aiming for a "minimal population" strikes me as much the worse and riskier of the two options, for both intrinsic moral reasons and instrumental ones like misjudging / undershooting the minimally sustainable level.
Your executive summary (quoted below) appears to outright assert that quantification is "harmful" and "results in poor decision making". I don't think those claims are well-supported.
If you paint a picture that focuses only on negatives and ignores positives, it's apt to be a very misleading picture. There may be possible ways to frame such a project so that it comes off as just "one piece of the puzzle" rather than as trying to bias its readership towards a negative judgment. But it's an inherently risky/difficult undertaking (prone to moral misdirection)...
Looking at EA’s history can show us strong and in many cases negative influence from utilitarian ideas.
It also shows strong and in vastly more cases positive influence from (what you call) "utilitarian" ideas (but really ought to be more universal--ideas like that it is better to do more good than less, and that quantification can help us to make such trade-offs on the basis of something other than mere vibes).
Unless there's some reason to think that the negative outweighs the positive, you haven't actually given us any reason to think that "utilitarian in...
Mostly just changing old habits, plus some anticipated missing of distinctive desired tastes. It's not an unreasonable ask or anything, but I'd much rather just donate more. (In general, I suspect there's insufficient social pressure on people to increase our donations to good causes, which also shouldn't be "so effortful", and we likely overestimate the personal value we get from marginal spending on ourselves.)
I don't understand the relevance of the correlation claim. People who care nothing for animals won't do either. But that doesn't show that there aren't tradeoffs in how to use one's moral efforts on the margins. (Perhaps you're thinking of each choice as a binary: "donate some" Y/N + "go vegan" Y/N? But donating isn't binary. What matters is how much you donate, and my suggestion is that any significant effort spent towards adopting a vegan diet might be better spent on further increasing one's donations. It depends on the details, of course. If you find a...
My main confusion with your argument is that I don't understand why donations don't also count as "personal ethics" or as "visible ethical action" that could likewise "ripple outward" and be replicated by others to good effect. (I also think the section on "equity" fundamentally confuses what ethics should be about. I care about helping beneficiaries, not setting up an "equitable moral landscape" among agents, if the latter involves preventing the rich from pursuing easy moral wins because this would be "unfair" to those who can't afford to donate.)
One mor...
The authors discuss this a bit. They note that even "higher fertility" subcultures are trending down over time, so it's not sufficiently clear that anyone is going to remain "above replacement" in the long run. That said, this does seem the weakest point for thinking it an outright extinction risk. (Though especially if the only sufficiently high-fertility subcultures are relatively illiberal and anti-scientific ones - Amish, etc. - the loss of all other cultures could still count as a significant loss of humanity's long-term potential! I hope it's OK to note this; I know the mods are wary that discussion in this vicinity can often get messy.)
I wrote "perhaps the simplest and most probable extinction risk". There's room for others to judge another more probable. But it's perfectly reasonable to take as most probable the only one that is currently on track to cause extinction. (It's hard to make confident predictions about any extinction risks.) I think it would be silly to dismiss this simply due to uncertainty about future trends.
I'd guess that (for many readers of the book) less air travel outweighs "buying more" furniture and kids toys, at least. But the larger point isn't that the change is literally zero, but that it doesn't make a sufficiently noticeable change to near-term emissions to be an effective strategy. It would be crazy to recommend a DINK lifestyle specifically in order to reduce emissions in the next 25 years. Like boycotting plastic straws or chatgpt.
Updated to add the figure from this paper, which shows no noticeable difference by 2050 (and little difference even...
As a general rule, it isn't necessary to agree on the ideal target in order to agree directionally about what to do on present margins. For example, we can agree that it would be good to encourage more effective giving in the population, without committing to the view (that many people would "personally disagree" with) that everyone ought to give to the point of marginal utility, where they are just as desperate for their marginal dollar as their potential beneficiaries are.
The key claim of After the Spike is that we should want to avoid massive depopulati...
Everyone has fundamental assumptions. You could imagine someone who disagrees with yours calling them "just vibes" or "presuppositions", but that doesn't yet establish that there's anything wrong with your assumptions. To show an error, the critic would need to put forward some (disputable) positive claims of their own.
The level of agreement just shows that plenty of others share my starting assumptions.
If you take arguments to be "circular" whenever a determined opponent could dispute them, I have news for you: there is no such thing as an argument that l...
I agree it's often helpful to make our implicit standards explicit. But I disagree that that's "what we're actually asking". At least in my own normative thought, I don't just wonder about what meets my standards. And I don't just disagree with others about what does or doesn't meet their standards or mine. I think the most important disagreement of all is over which standards are really warranted.
On your view, there may not be any normative disagreement, once we all agree about the logical and empirical facts. I think it's key to philosophy that the...
Actually have high integrity, which means not being 100% a utilitarian/consequentialist
Sorry for the necro-reply, but just saw this and wanted to register that I think a 100% utilitarian/consequentialist can still genuinely have high integrity. (I think people are generally quite confused about what a fitting consequentialist mindset looks like. It absolutely is not: "do whatever I naively estimate will maximize expected value, without regard for trustworthiness etc.") See, e.g., Naïve Instrumentalism vs Principled Proceduralism.
Ok, thanks for expanding upon your view! It sounds broadly akin to how I'm inclined to address Pascal's Mugging cases (treat the astronomical stakes as implying proportionately negligible probability). Astronomical stakes from x-risk mitigation seems much more substantively credible to me, but I don't have much to add at this point if you don't share that substantive judgment!